PaperlayerView exact revision on arXiv

Source-linked AI summary

GPTIPS 2: an open-source software platform for symbolic data mining

Dominic P. Searson

arXiv:1412.4690v2cs.MScs.NE

TL;DR

GPTIPS addresses the limited accessibility of symbolic data-mining software for discovering interpretable models from complex data. It combines multigene genetic programming with an extensible MATLAB platform and visual analytics, with reported applications showing strong predictive performance and model simplicity. GPTIPS 2 also targets compact models by identifying and removing horizontal bloat, while future out-of-the-box classification support remains a stated boundary.

  • Problem

    Existing symbolic data-mining software is relatively scarce, and available tools may be expensive, proprietary, closed source, or difficult to configure effectively.

  • Method

    GPTIPS uses multigene genetic programming to evolve symbolic models within a free, open-source MATLAB platform with pluggable code and visual-analytic tools.

  • Results

    GPTIPS using MGGP has been reported to outperform neural networks and support vector machines across many problem domains in predictive performance and model simplicity.

  • Takeaways & Limitations

    Gene-centric analysis can remove horizontal bloat and identify accurate, user-tailored, compact symbolic models from multigene regression results.

  • Takeaways & Limitations

    GPTIPS 2 mainly focuses on symbolic regression, while improved out-of-the-box symbolic-classification functionality is reserved for future updates.

Abstract

from arXiv · show

GPTIPS is a free, open source MATLAB based software platform for symbolic data mining (SDM). It uses a multigene variant of the biologically inspired machine learning method of genetic programming (MGGP) as the engine that drives the automatic model discovery process. Symbolic data mining is the process of extracting hidden, meaningful relationships from data in the form of symbolic equations. In contrast to other data-mining methods, the structural transparency of the generated predictive equations can give new insights into the physical systems or processes that generated the data. Furthermore, this transparency makes the models very easy to deploy outside of MATLAB. The rationale behind GPTIPS is to reduce the technical barriers to using, understanding, visualising and deploying GP based symbolic models of data, whilst at the same time remaining highly customisable and delivering robust numerical performance for power users. In this chapter, notable new features of the latest version of the software are discussed with these aims in mind. Additionally, a simplified variant of the MGGP high level gene crossover mechanism is proposed. It is demonstrated that the new functionality of GPTIPS 2 (a) facilitates the discovery of compact symbolic relationships from data using multiple approaches, e.g. using novel gene-centric visualisation analysis to mitigate horizontal bloat and reduce complexity in multigene symbolic regression models (b) provides numerous methods for visualising the properties of symbolic models (c) emphasises the generation of graphically navigable libraries of models that are optimal in terms of the Pareto trade off surface of model performance and complexity and (d) expedites real world applications by the simple, rapid and robust deployment of symbolic models outside the software environment they were developed in.

1 Introduction

Symbolic data mining evolves symbolic equations from data, offering interpretable and deployable models when underlying relationships are complex or unknown. GPTIPS addresses limited accessible software through a free, open-source MATLAB platform for multigene symbolic regression.

  • Symbolic data mining includes symbolic regression, classification, and optimisation activities that generate equations or predictions from data.
  • Symbolic regression automatically evolves model structure and parameters, selecting inputs and capturing nonlinear behaviour instead of requiring a predefined model structure.
  • Symbolic regression models use a symbolic nonlinear function to predict an output from input variables, with GPTIPS supporting compact mathematical representations.
  • Interpretable symbolic equations are attractive for multivariate nonlinear systems because they can support human understanding and deployment outside specialised modelling environments.
  • GPTIPS was created to reduce technical barriers to symbolic data mining for researchers without deep computing or machine-learning expertise.
  • GPTIPS is free, open source, MATLAB-based, customisable, and intended to support model discovery, understanding, and deployment.

2 GPTIPS 2 – Overview

GPTIPS 2 uses multigene genetic programming to evolve symbolic models and adds tooling for extensible modelling, analysis, visualisation, and deployment. Its documented applications include strong predictive performance, model simplicity, and accurate compact models across diverse domains.

  • Reported applications include models up to 70% better than existing empirical correlations and biological-network model accuracy of around 99%.
  • Multigene genetic programming evolves data structures containing multiple trees, or genes, whose weighted combination forms a GPTIPS regression model.
  • GPTIPS has a pluggable architecture that lets users add objective or fitness functions for tasks such as symbolic classification and optimisation without modifying core code.
  • Multigene symbolic regression can evolve compact, accurate models and automatically select features with more than 1500 input variables.
  • 2.1 GPTIPS Feature Overview: GPTIPS 2 supports parallel fitness and complexity calculations, separate training, validation, and test reporting, extensible tree functions, symbolic post-analysis, and termination criteria.
  • 2.1 GPTIPS Feature Overview: The platform provides model-tree reports, interactive Pareto-front libraries sortable by R2 and complexity, and REC curves for comparing selected models.

3 Multigene Symbolic Regression and MGGP – Overview and Mathematical context

The section introduces multigene symbolic regression as an MGGP-based approach that evolves additive combinations of trees, with coefficients estimated by least squares. It also describes the evolutionary workflow and rate-based high-level crossover used in GPTIPS 2.

  • Naïve and scaled symbolic regression: Unlike naïve symbolic regression, scaled symbolic regression adds a bias and scaling coefficient to one tree output.The coefficients are estimated by linear least squares from the tree output and response data.
  • Multigene symbolic regression: MGGP evolves multiple gene trees whose weighted linear combination forms a symbolic regression model.Each gene is a tree, and GPTIPS evolves the trees comprising the additive model terms.
  • Model complexity: Depth restrictions, Pareto tournaments, and expressional complexity promote compact models with linearly separable terms suitable for symbolic simplification.Increasing Gmax can capture more nonlinear behaviour but also risks overfitting and horizontal bloat.
  • Multigene symbolic regression: A multigene model predicts y as ŷ = b0 + b1 t1 + … + bG tG, with one bias and one coefficient per gene.The gene outputs are assembled into a response matrix, allowing the model to be written as ŷ = Gb and fitted by least squares.
  • Coefficient estimation: GPTIPS uses the Moore-Penrose pseudo-inverse when gene-response columns are collinear, including when an individual contains duplicate genes.The resulting coefficients support RMSE-based fitness evaluation, although RMSE can be changed to another measure such as MSE.
  • MGGP evolution: GPTIPS 2 simplifies high-level crossover by exchanging randomly selected genes between parent individuals, subject to the Gmax constraint.The rate-based mechanism can create new genes, change model gene counts, and randomly delete genes when an offspring exceeds Gmax.

4 Using GPTIPS

GPTIPS 2 combines command-line workflows with interactive tools for filtering, inspecting, comparing, reporting, and exporting multigene regression models. Its analyses emphasize Pareto trade-offs between predictive performance and model complexity across training, validation, and test data.

  • Running GPTIPS: Users configure runs through MATLAB files, execute them with rungp, and retain the resulting population and analysis data in the gp structure.Unspecified parameters receive GPTIPS defaults.
  • Exploratory post-run analyses: The summary function provides a compact overview of a completed GPTIPS run.It is one of the exploratory post-run visualisation tools.
  • Exploratory post-run analyses: The population browser plots models by (1 − R2) and expressional complexity, highlighting Pareto-front models and the training-data R2 leader.Models in the lower-left region combine high R2 with low complexity and are usually the main candidates for further investigation.
  • Exploratory post-run analyses: The paretoreport function creates a sortable standalone HTML table of Pareto models, including simplified equations, training-data R2, and expressional complexity.Sorting by performance or complexity helps identify promising models for detailed investigation.
  • Population filtering: Users can filter populations by criteria such as R2 thresholds, required or excluded variables, complexity, and Pareto-front membership before applying further analyses.Filtered populations remain compatible with GPTIPS command-line and visual-analysis tools.
  • Model performance and export: The runtree function evaluates selected models on available training, validation, and test data, while reports and REC curves support detailed model comparison.GPTIPS also exports models as standalone MATLAB files or symbolic objects that can be converted to C code.

5 Reducing Model Complexity using Gene Analysis

GPTIPS 2 addresses horizontal bloat in multigene regression by helping users identify and remove genes with little predictive value. Its gene-centric tools expose model complexity, gene equations, and performance changes to support informed simplification and domain-tailored models.

  • 5.1 Horizontal Model Bloat: Horizontal bloat adds performance-neutral or marginally useful genes to multigene models, making many resulting terms undesirable.The problem parallels non-regularised multiple linear regression, where training R2 can increase despite noise-fitting or poor generalisation.
  • 5.2 Unique Gene Analysis: GPTIPS 2 uses gene-centric visualisation to identify genes that can be removed while causing relatively small changes in predictive performance.The tool displays gene equations and the R2 value that would result if a model gene were removed.
  • 5.2 Unique Gene Analysis: Visualising individual genes helps users understand which model terms contribute more or less to predictive ability and tailor models using domain knowledge.This supports removal of terms that conflict with knowledge of the underlying system.
  • 5.2 Unique Gene Analysis: The uniquegenes workflow extracts gene trees, removes duplicate genotypes, simplifies symbolic phenotypes, and assigns IDs to the remaining unique genes.Different genotypes can simplify to the same symbolic phenotype.
  • 5.2 Unique Gene Analysis: The genebrowser compares genes in a selected model with genes elsewhere in the population and reports the effect of deleting or adding each gene.Blue bars represent model genes; orange bars represent genes absent from the current model.
  • 5.2 Unique Gene Analysis: In the Cherkassky-function example, the selected model contains 3 genes with training R2 0.99982 and complexity 65, and gene 47 is identified as the top removal candidate.The resulting model can be regenerated with genes2gpmodel and inspected or exported using the same tools as other population models.

6 Conclusions

GPTIPS 2 is presented as an open-source platform for discovering, understanding, and deploying simple symbolic models from nonlinear, high-dimensional data. The chapter highlights gene-centric analysis for reducing horizontal bloat while noting that out-of-the-box symbolic-classification support remains a future update.

  • 6 Conclusions: GPTIPS 2 targets non-experts while providing tools for automatically discovering, understanding, and deploying symbolic models from nonlinear, high-dimensional data.The platform is also intended to remain extensible for researchers developing additional symbolic data-mining applications.
  • 6 Conclusions: The chapter focuses largely on symbolic regression, while improved out-of-the-box functionality for symbolic classification is planned for future GPTIPS 2 updates.Researchers can add their own code for symbolic classification and symbolic optimisation.
  • 6 Conclusions: GPTIPS 2 introduces gene-centric visual analytics for identifying and removing unnecessary complexity in multigene regression models.The stated outcome is accurate, user-tailored, compact, data-driven symbolic models.
Loading 1412.4690v2…