PaperlayerView exact revision on arXiv

Source-linked AI summary

VizML: A Machine Learning Approach to Visualization Recommendation

Kevin Z. Hu, Michiel A. Bakker, Stephen Li, Tim Kraska, César A. Hidalgo

arXiv:1808.04819v1cs.HCcs.AIcs.LG

TL;DR

Visualization tools can be inaccessible because basic designs often require manual specification. VizML learns visualization design choices from one million dataset–visualization pairs, and its predictions achieve high accuracy while performing comparably to humans on a crowdsourced consensus benchmark.

  • Problem

    Manual visualization specification creates a steep barrier for analysts without the time or background to learn sophisticated tools.

  • Method

    VizML trains neural-network models to predict visualization design choices from a large corpus of datasets and associated visualizations.

  • Results

    VizML achieves ∼70−95% accuracy on design-choice prediction and performs comparably to humans on a crowdsourced consensus visualization-type test.

  • Takeaways & Limitations

    Dataset structural and statistical properties influence visualization choices, and ML can capture and leverage these properties in visualization recommendation.

  • Takeaways & Limitations

    The corpus is biased toward Plotly users, interfaces, and datasets, limiting the approach’s demonstrated generalizability.

Abstract

from arXiv · show

Data visualization should be accessible for all analysts with data, not just the few with technical expertise. Visualization recommender systems aim to lower the barrier to exploring basic visualizations by automatically generating results for analysts to search and select, rather than manually specify. Here, we demonstrate a novel machine learning-based approach to visualization recommendation that learns visualization design choices from a large corpus of datasets and associated visualizations. First, we identify five key design choices made by analysts while creating visualizations, such as selecting a visualization type and choosing to encode a column along the X- or Y-axis. We train models to predict these design choices using one million dataset-visualization pairs collected from a popular online visualization platform. Neural networks predict these design choices with high accuracy compared to baseline models. We report and interpret feature importances from one of these baseline models. To evaluate the generalizability and uncertainty of our approach, we benchmark with a crowdsourced test set, and show that the performance of our model is comparable to human performance when predicting consensus visualization type, and exceeds that of other ML-based systems.

1 INTRODUCTION

VizML frames visualization creation as a set of design choices that can be learned from datasets and associated visualizations. Its models achieve high predictive accuracy and perform comparably to humans on a crowdsourced visualization-type benchmark.

  • Motivation: Manual specification makes visualization inaccessible to domain experts lacking time or technical background.
  • Motivation: Visualization recommendation can prioritize speed and breadth for common exploratory tasks by using dataset properties to suggest designs.
  • Approach: VizML formulates recommendation as learning design choices from a corpus of datasets and corresponding visualizations.
  • Results: Neural networks trained on 60% of the corpus achieve ∼70−95% accuracy on design-choice prediction in a separate 20% test set.The models exceed four simpler baselines, which themselves outperform random chance.
  • Results: VizML performs comparably to Plotly users and Mechanical Turk workers on a crowdsourced consensus test and outperforms other ML-based models.
  • Scope: Independent design-choice recommendations may be incompatible and may not maximize overall visualization effectiveness.A complete visualization requires modeling dependencies among the recommendation models.

3 RELATED WORK

Prior visualization recommenders rely mainly on rules or smaller, controlled ML datasets. VizML instead predicts design choices from a larger and more diverse corpus, improving validation and interpretability while retaining important scope limitations.

  • Rule-based systems: Rule-based recommenders use perceptual and expert-guided rules to generate, rank, and select visualizations.
  • Rule-based systems: Rule-based systems face difficult nonlinear encoding, expert-authoring costs, cold starts, and combinatorial recommendation growth.
  • ML-based systems: Existing ML systems classify or rank visualizations, generate specifications end-to-end, or learn weights for formal constraints rather than predict design choices.
  • VizML distinction: Predicting design choices makes VizML easier to quantitatively validate, interpretable through feature importances, and easier to integrate into visualization systems.
  • VizML distinction: VizML uses a corpus orders of magnitude larger than DeepEye and Data2Vis, enabling larger feature sets and higher-capacity models.
  • VizML distinction: VizML’s corpus contains diverse real analyst datasets and visualizations, unlike systems trained on few datasets or rule-generated examples in controlled settings.
  • Limitations: VizML recommends visual encodings but not data queries, and the paper does not create a user-facing application.

4 DATA

The study builds a large Plotly corpus, cleans it into dataset–visualization pairs, and represents datasets through column-, pairwise-, and dataset-level features alongside extracted visualization design choices.

  • 4.1 Collection and Cleaning: Plotly visualizations use traces for data mappings and a layout dictionary for display configuration.The API associates each visualization with data, specification, and layout objects.
  • 4.1 Collection and Cleaning: The corpus contains 2,359,175 visualizations collected over approximately 2.5 years, with 1,989,068 parsed without error.The collection spans 2015-07-17 to 2018-01-06 and is called the Plotly corpus.
  • 4.2 Data Description: Because 98.32% of visualizations use one source dataset and over 90% use all source columns, the corpus does not address data query selection, transformation, or aggregation.Only 0.16% of 13,321,598 traces contain transformations or aggregations.
  • 4.3 Feature Extraction: The feature representation includes 81 single-column features, 30 pairwise-column features, and 841 dataset-level aggregate features.Features describe dimensions, types, values, names, column relationships, and aggregated dataset properties.
  • 4.4 Design Choice Extraction: Design choices are extracted from visualization traces, including mark type, X- or Y-column encoding, axis sharing, and visualization type.Visualization-level choices aggregate encoding-level choices across traces.

5 METHODS

VizML trains models to predict visualization design choices from processed dataset features, using a workflow that constructs features and labels, fits task-specific models, and produces recommended choices.

  • 5 METHODS: Raw features are one-hot encoded, clipped at the 1st and 99th percentiles, mean-imputed or mode-imputed, centered, and scaled.Categorical missing values use the mode; numeric missing values use the mean.
  • 5 METHODS: The final corpus used for modeling contains 119,815 deduplicated datasets and 287,416 columns.Only one randomly selected dataset per user was retained after exact deduplication.
  • 5 METHODS: The workflow converts raw Plotly data into features, extracts design choices, trains task-specific models, and outputs predicted choices.The workflow is summarized as data source, deduplicated pairs, features and extracted choices, models, and recommendations.

2. Has Shared Axis [HSA]: 2-class

The shared-axis task predicts whether all chart traces share one axis, using features describing individual columns rather than complete datasets.

  • 2. Has Shared Axis [HSA]: 2-class: The HSA task predicts whether traces in a chart share one axis, either X or Y.The task is defined as a 2-class prediction problem.
  • 2. Has Shared Axis [HSA]: 2-class: The encoding-level tasks consider each column independently instead of alongside other columns in the same dataset.This bag-of-columns formulation accounts for the effect of column order.
  • 2. Has Shared Axis [HSA]: 2-class: Mark type is a separate encoding-level task that predicts the visual mark representing a column.Its outcomes include scatter, line, bar, box, histogram, and heatmap.

2. Is Shared X-axis or Y-axis [ISA]: 2-class

The ISA task predicts whether a column is encoded on the X- or Y-axis, within a broader evaluation that compares neural networks with simpler baselines across design-choice tasks.

  • 2. Is Shared X-axis or Y-axis [ISA]: 2-class: ISA is a 2-class task asking whether a column is encoded on the X-axis or the Y-axis.It is one of three encoding-level prediction tasks based on individual-column features.
  • 5 METHODS: The models evaluated include a feedforward neural network, naive Bayes, K-nearest neighbors, logistic regression, and random forests.The neural network is the primary model and the other four are baselines.
  • 5 METHODS: Results are averaged over 5-fold cross-validation, with each corpus sample included in exactly one test set.Accuracy is used as the performance measure.
  • 5 METHODS: The neural network is evaluated with four incrementally expanded feature sets, while baseline models use the full feature set.The neural-network sets add Dimensions, Types, Values, and Names categories in sequence.
  • 5 METHODS: The evaluation reports prediction accuracies for two visualization-level and three encoding-level tasks.The results table covers all five models and feature-category settings.

6 EVALUATING PREDICTION PERFORMANCE

Across seven prediction tasks, neural networks achieved the highest accuracies, while simpler models often performed comparably and exceeded random guessing.

  • Neural networks achieved the highest mean accuracies across all seven prediction tasks.Every model exceeded the random-guessing baseline.
  • Model performance generally improved in the order NB < KNN < LR ≈RF < NN.
  • RF and LR were not significantly less accurate than NN in most tasks.
  • Value-based features contributed more accuracy than type-based features, potentially because they are more numerous or overlap with type information.

7 INTERPRETING FEATURE IMPORTANCES

Feature-importance analysis identifies dataset dimensionality, column type, distributional statistics, orderedness, and scale-related measures as influential predictors across visualization and encoding tasks.

  • Interpreting feature importances: Feature importances were calculated to interpret models, validate the feature-extraction pipeline, and connect features to prior literature.They can also inform guidelines, rule-based systems, and feature selection.
  • Interpreting feature importances: Mean decrease impurity was used to identify the top ten features for the best-performing random forest models.The method was chosen for interpretability and stability across runs.
  • Feature groups: Dimensionality features, including row count and column count, were important for prediction tasks.Column length was the second most important feature for predicting line versus bar traces.
  • Feature groups: Column-type features were consistently important, including string-column presence for predicting bar versus line charts.
  • Feature groups: Statistical features such as Gini, entropy, skewness, and kurtosis were important across tasks, while mean and variance were less important.The results indicate the potential value of higher-order distributional shape information.
  • Feature groups: Orderedness measures, specifically sortedness and monotonicity, were important for many tasks.Sortedness is defined using the correlation between sorted and unsorted column values.
  • Feature groups: Linear and logarithmic space sequence coefficients, which approximate scale of variation, were important features.A column is classified as linear or logarithmic when its coefficient is ≤10^-3.
  • Implications and limitations: Rule-based systems may need broader feature sets because capturing nonlinear task dependencies and complex feature relationships remains difficult.Most extracted features can nevertheless be computed with inexpensive linear operations.

8 BENCHMARKING WITH CROWDSOURCED EFFECTIVENESS

The paper benchmarks visualization recommendations against crowdsourced consensus, using a continuous effectiveness score that captures ambiguity and supports uncertainty estimation. VizML performs comparably to human-based predictors and outperforms other machine-learning systems on two- and three-class visualization-type tasks.

  • 8.1 Modeling and Measuring Effectiveness: A continuous effectiveness score allows multiple visualization choices to receive non-zero or maximal scores when they are equally effective.The score normalizes each choice probability by the maximum probability, so equally frequent choices can both receive the maximum score.
  • 8.1 Modeling and Measuring Effectiveness: The crowdsourced benchmark serves to estimate model uncertainty, test generalization from the Plotly corpus, and compare VizML with users and other predictors.Confidence intervals are established through bootstrap samples of Mechanical Turk votes.
  • 8.2 Benchmarking Test Set: The benchmark uses 99 datasets visualized as bar, line, and scatter charts after excluding incomplete datasets and cases where all three types could not encode the information.The final set contains 33 datasets originally represented by each chart type.
  • 8.4 Benchmarking Procedure: The benchmark compares Plotly users, individual Mechanical Turk workers, VizML, and baseline predictors using the consensus-adjusted recommendation score.The score rewards predictions according to the normalized effectiveness of the predicted visualization type and uses 95% bootstrap confidence intervals.
  • 8.5 Benchmarking Results: Gini distributions are not concentrated at either consensus extreme, supporting CARS as a soft measure rather than hard accuracy for ambiguous visualization preferences.Lower Gini values indicate weaker consensus and a more ambiguous ideal visualization type.
  • 8.5 Benchmarking Results: VizML achieves CARS of 88.96 ± 1.66 versus 86.66 ± 5.38 for Mechanical Turkers and 90.35 ± 1.85 for Plotly users in the two-class case.It significantly outperforms Data2Vis at 75.61 ± 2.44 and DeepEye at 79.12 ± 4.33.
  • 8.5 Benchmarking Results: VizML achieves CARS of 81.18 ± 2.39 in the three-class case, slightly higher but within error bars than Mechanical Turkers at 79.28 ± 4.66 and Plotly users at 79.58 ± 2.44.Data2Vis and DeepEye exceed Random, but remain below VizML and human predictors.

9 DISCUSSION

VizML achieves high accuracy on five visualization-design prediction tasks and performs comparably to humans on a crowdsourced consensus benchmark, while revealing important scope limitations.

  • VizML neural networks achieve high test accuracies on five prediction tasks relative to random guessing and simpler classifiers.
  • On a crowdsourced consensus test set, neural-network performance is comparable to individual humans.
  • The corpus and approach are biased toward Plotly users, interfaces, plot types, and smaller datasets despite deduplication.
  • The study covers only a subset of visualization-recommendation pipeline tasks, whereas an ideal user-facing tool would support the entire pipeline.
  • The results support regularities in how humans visualize data, while crowd disagreement indicates that visualization design choices also contain subjective elements.

10 FUTURE RESEARCH DIRECTIONS

Future work targets stronger evaluation resources, broader data and representations, safer recommendation practices, and integration of separate predictions into an end-to-end system.

  • Public Training and Benchmarking Corpuses: A standard public benchmark is needed because its absence impedes bootstrapping recommender systems and comparing approaches.
  • Diverse Data Sources: Plotly data constrain VizML to clean datasets and visualizations encoding all columns, excluding upstream feature selection and transformation tasks.
  • Transfer Learning: Unsaturated learning curves indicate that more training data could improve the approach, motivating transfer learning as another strategy.
  • Representation Learning: Representation learning from raw data could bypass feature engineering and derive hand-engineered features and additional ones.
  • Unsupervised Learning: Unsupervised methods such as t-SNE and UMAP reveal non-trivial structure in dataset space, suggesting another route to end-to-end recommendation.
  • Addressing the Multiple Comparisons Problem: Opaque ML recommenders exacerbate multiple-comparisons concerns because their implicit comparisons are difficult to track.
  • Integrating Prediction Tasks into Pipeline Model: An end-to-end model would combine separate prediction-task outcomes into ranked recommendations, but predicted outcomes can sometimes be inconsistent.

APPENDIX A FEATURES AND AGGREGATIONS

The appendix identifies a single-column feature representation and references a table of feature aggregation functions.

  • Each column is described by 81 single-column features.
  • The features describe individual columns’ dimensions, types, values, and names.
  • Table 4 is titled “Features and aggregation functions.”
Loading 1808.04819v1…