Source-linked AI summary
pymoo: Multi-objective Optimization in Python
Julian Blank, Kalyanmoy Deb
TL;DR
Existing Python optimization frameworks provide incomplete coverage for multi-objective tasks, motivating a more comprehensive solution. The paper introduces pymoo, a customizable framework with algorithms, test problems, analysis, and practical tooling, and presents its architecture and constrained optimization workflow. Its implementations are intended to provide benchmark results, while the framework remains open to expansion with more algorithms, problems, and performance-analysis capabilities.
Problem
Existing optimization frameworks only partially support multi-objective optimization and do not comprehensively cover complete multi-objective tasks.
Method
pymoo combines customizable optimization algorithms with test problems, automatic differentiation, parallel evaluation, visualization, performance indicators, and multi-criteria decision making.
Results
pymoo covers multiple aspects of multi-objective optimization through its Problems, Optimization, and Analytics modules, including visualization and multi-criteria decision making.
Takeaways & Limitations
The framework provides users with multiple options for key aspects of multi-objective optimization and supports selecting a solution from an obtained solution set.
Takeaways & Limitations
The framework currently offers only a few basic performance metrics and plans further additions of algorithms, test problems, single-objective methods, and automated statistics.
Abstract
from arXiv · showhide
Python has become the programming language of choice for research and industry projects related to data science, machine learning, and deep learning. Since optimization is an inherent part of these research fields, more optimization related frameworks have arisen in the past few years. Only a few of them support optimization of multiple conflicting objectives at a time, but do not provide comprehensive tools for a complete multi-objective optimization task. To address this issue, we have developed pymoo, a multi-objective optimization framework in Python. We provide a guide to getting started with our framework by demonstrating the implementation of an exemplary constrained multi-objective optimization scenario. Moreover, we give a high-level overview of the architecture of pymoo to show its capabilities followed by an explanation of each module and its corresponding sub-modules. The implementations in our framework are customizable and algorithms can be modified/extended by supplying custom operators. Moreover, a variety of single, multi and many-objective test problems are provided and gradients can be retrieved by automatic differentiation out of the box. Also, pymoo addresses practical needs, such as the parallelization of function evaluations, methods to visualize low and high-dimensional spaces, and tools for multi-criteria decision making. For more information about pymoo, readers are encouraged to visit: https://pymoo.org
1. Introduction
pymoo is introduced as a Python framework addressing multi-objective optimization needs with a broad, customizable set of tools. The paper demonstrates its use and explains its architecture, documentation, and supporting infrastructure.
- pymoo addresses the need for multi-objective optimization in Python while covering multiple aspects of the optimization process.
- The framework provides single-, multi-, and many-objective test problems, automatic differentiation for gradients, parallelized evaluation, performance indicators, visualization, and multi-criteria decision making.
- Its modular implementation lets users assemble algorithms from interchangeable sub-modules and incorporate domain knowledge through custom implementations.
- A starter’s guide demonstrates a constrained bi-objective problem, while documentation explains algorithms, source code, and test problems.
- Documentation uses Sphinx, module correctness is checked with automatic unit testing, and most algorithms have been benchmarked against original implementations.
- The paper presents related frameworks, guides users through pymoo, and describes its architecture and modules in subsequent sections.
2. Related Works
Existing optimization frameworks span languages and differ in their coverage of multi-objective optimization and supporting features. Python alternatives offer varied combinations of algorithms, analysis, parallelization, modularity, and visualization.
- Framework selection involves multi-objective criteria, including feature coverage, usability, extendibility, programming language, parallelization, and visualization.
- jMetalPy ports jMetal to Python and combines traditional and dynamic optimization with post-analysis capabilities.
- PyGMO distributes massive optimization tasks across multiple CPUs using a generalized island-model paradigm for coarse-grained parallelization.
- Platypus provides state-of-the-art algorithms, experiment creation across algorithms, and post-analysis through metrics and visualization.
- DEAP targets evolutionary-computation prototyping rather than multi-objective optimization directly, but its modularity supports multi-objective algorithms and built-in parallelization.
- Inspyred focuses on bio-inspired computational intelligence and supports multi-objective algorithms through an NSGA-II example and modular implementation.
- Frameworks outside Python include PlatEMO in Matlab, MOEA and jMetal in Java, and jMetalCpp and PaGMO in C++.
3. Getting Started 1
The getting-started guide walks through installing pymoo, defining and implementing a constrained bi-objective problem, initializing NSGA-II, and running the optimization. In the example, the algorithm converges to a nearly optimal set of solutions.
- Getting Started: The guide covers installation, problem definition, optimization, and the procedure for solving a constrained multi-objective problem.
- Problem Definition: The example has two objectives, two inequality constraints, two variables bounded in [−2, 2], and no equality constraints.
- Problem Definition: The example converts maximization and greater-than-or-equal constraints into pymoo’s pure-minimization and less-than-or-equal formulation.
- Problem Implementation: pymoo represents optimization problems through a Problem subclass whose evaluation function returns objective values under key F and constraints under key G.
- Algorithm Initialization: NSGA-II uses a population of 40, generates 10 offspring per generation, checks duplicates, and runs for 40 generations with 440 function evaluations.The configuration uses real-random sampling, simulated binary crossover, and polynomial mutation.
- Optimization Results: The resulting solutions are nearly optimal and converge toward the analytically derived Pareto set and front.The design and objective spaces are shown separately in Figure 2.
4. Architecture
pymoo’s architecture organizes the framework into problems, optimization algorithms, and analytics, each composed of more detailed sub-modules. These modules support test problems, algorithm operators and constraint handling, and analysis of optimization results.
- Architecture Overview: The first architectural abstraction level contains optimization problems, algorithms, and analytics, with each module divided into multiple sub-modules.
- Problems: Problem modules provide single-, multi-, and many-objective test problems, automatic-differentiation gradients, and parallelization techniques.
- Optimization: Optimization modules configure evolutionary operators, constraint handling, decomposition methods, and termination criteria.
- Analytics: Analytics modules support visualization of design and objective spaces, performance indicators for convergence and diversity, and multi-criteria decision making.
5. Problems
pymoo provides categorized test problems, configurable outputs, automatic differentiation, and several strategies for parallelizing solution evaluations.
- Test problems: pymoo categorizes test problems as single-objective, multi-objective, or many-objective according to the number of objectives.The categories contain 1, 2–3, and more than 3 objectives, respectively.
- Implementations: Problem evaluation can return objective values, constraint violations, constraint functions, and corresponding gradients through selectable return values.The default outputs are objective values and, when applicable, constraint violations; adding prefixes such as dF or dG requests gradients.
- Gradients: Automatic differentiation is supported out of the box because pymoo implements all test problems using Autograd.The paper demonstrates gradient retrieval for objective functions and notes that constraint gradients can be requested similarly.
- Parallelization: Vectorized, threaded, and distributed evaluation strategies allow users to parallelize computationally expensive solution evaluations.Vectorization evaluates populations together, threading runs independent evaluations concurrently, and distributed evaluation can use multiple workers or clusters.
6. Optimization Module
pymoo builds algorithms from modular optimization components and supports customizable operators, termination criteria, and decomposition methods.
- Algorithms: Algorithms in pymoo are assembled from sub-modules, and variants can be initialized by changing parameters or combining components.The framework includes generic modules such as decomposition and termination criteria alongside evolutionary-computing modules.
- Operators: Available evolutionary operators include sampling, crossover, and mutation for real, integer, and binary variables.Sampling can use random methods or Latin-Hypercube Sampling, while mutation includes Polynomial Mutation and Bitflip mutation.
- Operators: Custom operators and variable types can be implemented and supplied to algorithms for repeated problem-solving or specialized representations.The paper gives trees and mixed variables as examples of custom variable types.
- Termination Criterion: Termination can use evaluations, iterations, performance changes, or movement in design or objective space across generations.One implemented criterion tracks the largest movement from a solution to its closest neighbour over the last k generations.
- Decomposition: Decomposition transforms multi-objective problems into many single-objective problems using methods including Weighted-Sum, Tchebysheff, ASF, AASF, and PBI.The paper notes that normalization is essential for decomposition and that Weighted-Sum cannot converge to non-convex Pareto-front regions.
7. Analytics
pymoo’s analytics module supports evaluating, visualizing, and selecting solutions from multi-objective optimization results. It includes performance indicators, customizable plots for different objective dimensions, and multi-criteria decision-making methods.
- Performance Indicators: GD measures convergence toward a Pareto front, whereas IGD measures the average distance from Pareto-front solutions to the obtained set.
- Performance Indicators: Hypervolume measures the dominated objective-space portion relative to a reference point and is Pareto compliant, but exact computation becomes intractable in higher dimensions.
- Visualization: pymoo provides customizable visualizations including scatter plots, pairwise scatter plots, parallel-coordinate plots, Radviz, star-coordinate plots, heatmaps, petal diagrams, and radar diagrams.
- Visualization: Radar diagrams display one solution using objective axes bounded by ideal and nadir polygons, with normalization available when objective ranges differ.
- Decision Making: pymoo supports decision-making through scalarization functions, pseudo-weights, and high trade-off detection for choosing among non-dominated solutions.
- Decision Making: The trade-off procedure can restrict comparisons to the k closest objective-space neighbors to reduce computational complexity.
8. Concluding Remarks
The paper introduces pymoo as a Python framework covering the workflow from installation and constrained bi-objective optimization to analytics and decision-making. The authors present modular components and regard the implemented procedures as benchmark results, while identifying planned extensions.
- Concluding Remarks: pymoo provides three core modules—Problems, Optimization, and Analytics—and demonstrates their use on a constrained bi-objective optimization problem.
- Concluding Remarks: The framework covers evolutionary operators, performance metrics, visualization techniques, and decision-making approaches for multi-objective optimization.
- Concluding Remarks: The authors consider the implemented procedures authentic and error-free, so their results should serve as benchmark results for those procedures.
- Future Work: Planned extensions include more algorithms and test problems, classical single-objective methods for multi-objective use, and automated performance-indicator statistics.
- Future Work: pymoo is presented as a collaborative framework where community contributions can expand its methods and help researchers advertise their approaches.