PaperlayerView exact revision on arXiv

Source-linked AI summary

Transfer Learning based Dynamic Multiobjective Optimization Algorithms

Min Jiang, Zhongqiang Huang, Liming Qiu, Wenzhen Huang, Gary G. Yen

arXiv:1612.06093v2cs.NE

TL;DR

Dynamic multiobjective optimization must track changing Pareto-optimal fronts, while many existing prediction-based methods assume identically distributed data across environments. Tr-DMOEA applies transfer learning to reuse past Pareto-solution knowledge, and tests of three algorithms on twelve benchmark functions report improved solution quality and robustness in almost all results.

  • Problem

    Many existing dynamic multiobjective methods assume identically distributed data across environments, although objective changes make tracking varying Pareto-optimal fronts challenging.

    Source: §1 · p. 3, §2.2 · p. 9

  • Method

    Tr-DMOEA uses transfer component analysis to map data distributions into a latent space and reuse past Pareto-solution knowledge to generate an initial population for a new environment.

    Source: §1 · p. 3, §3 · p. 10

  • Results

    On twelve benchmark functions, almost all experimental results showed improved solution quality and algorithm robustness for transfer-learning-enhanced NSGA-II, MOPSO, and RM-MEDA.

    Source: §5 · p. 27

  • Takeaways & Limitations

    The authors present transfer learning as a new avenue for designing effective and efficient evolutionary algorithms for dynamic multiobjective optimization.

    Source: §5 · p. 27

    The experiments did not explicitly tune parameters separately for each algorithm and test function.

    Source: §4.2.1 · p. 23

Abstract

from arXiv · show

One of the major distinguishing features of the dynamic multiobjective optimization problems (DMOPs) is the optimization objectives will change over time, thus tracking the varying Pareto-optimal front becomes a challenge. One of the promising solutions is reusing the "experiences" to construct a prediction model via statistical machine learning approaches. However most of the existing methods ignore the non-independent and identically distributed nature of data used to construct the prediction model. In this paper, we propose an algorithmic framework, called Tr-DMOEA, which integrates transfer learning and population-based evolutionary algorithm for solving the DMOPs. This approach takes the transfer learning method as a tool to help reuse the past experience for speeding up the evolutionary process, and at the same time, any population based multiobjective algorithms can benefit from this integration without any extensive modifications. To verify this, we incorporate the proposed approach into the development of three well-known algorithms, nondominated sorting genetic algorithm II (NSGA-II), multiobjective particle swarm optimization (MOPSO), and the regularity model-based multiobjective estimation of distribution algorithm (RM-MEDA), and then employ twelve benchmark functions to test these algorithms as well as compare with some chosen state-of-the-art designs. The experimental results confirm the effectiveness of the proposed method through exploiting machine learning technology.

1 Department of Cognitive Science and Technology, Xiamen University, China · 2 Institute of Automation of the Chinese Academy of Sciences, China · 1 Introduction

Because changing Pareto-optimal fronts can make past and newly generated samples distributionally different, the paper proposes integrating transfer learning into evolutionary multiobjective algorithms to reuse past experience for faster re-optimization.

1 Introduction

  • DMOP objectives vary across time or environments, making Pareto-front tracking difficult; dynamic portfolio optimization is one real-world example.In deregulated electricity markets, changing prices and other factors can require hourly optimization.Source: §1 · p. 2
  • Prior prediction-based methods combine machine learning with evolutionary algorithms, including memory-based regression and Markov models, motion information, and a Kalman-extended genetic algorithm [49].Source: §1 · pp. 2–3
  • Reusing past solutions implicitly assumes training and predicted samples are identically distributed, but changing Pareto fronts can produce distribution mismatch that challenges traditional machine learning.Source: §1 · p. 3
  • Transfer learning permits different training and testing distributions, making it a tool for addressing the IID limitation in prediction-based dynamic multiobjective optimization.Source: §1 · p. 3
  • The proposed approach uses transfer component analysis to transfer knowledge about finding Pareto-optimal solutions—not the population—to create an initial pool for the next environment.The domain-adaptation approach is designed for incorporation into evolutionary multiobjective algorithms.Source: §1 · pp. 3–4
  • The paper focuses on quickly re-optimizing after dynamic changes have been detected, leaving change detection and identification outside its scope.Source: §1 · p. 4
  • Integrating transfer learning with classical multiobjective evolutionary algorithms preserves their advantages and improves search efficiency through reuse of past experience.The paper also reports that its experiments validate the population's importance for tracking dynamic optima, as supported theoretically in [14].Source: §1 · p. 4
  • The paper plans to test the approach with NSGA-II, MOPSO, and RM-MEDA on the IEEE CEC 2015 benchmark problems.Source: §1 · p. 4

2 Preliminary Studies and Related Research

The section defines dynamic multiobjective optimization in terms of time-varying objectives and Pareto-optimal sets and fronts, then reviews methods for tracking changes. It motivates transfer learning with the concern that prediction data may not be IID and presents a framework intended to reuse historical information without requiring IID data.

2.1 Dynamic Multi-objective Optimazation

  • A DMOP minimizes a vector of objectives over bounded decision variables as time or environment changes; its DPOS contains nondominated decisions, and the DPOF contains their objective vectors.An ideal algorithm should track the changing Pareto-optimal front while maintaining solution diversity.Source: §2.1 · p. 5

2.2 Related Works

  • Prior DMOP algorithms are grouped into increasing or maintaining diversity, memory-based, multi-population, and prediction-based methods.The review describes prediction-based methods as using existing information, often with machine learning, to anticipate environmental changes and adapt in advance.Source: §2.2 · pp. 6, 8
  • Diversity methods inject or relocate individuals after changes, while memory methods reuse archived solutions; their effectiveness can be limited by uncontrolled mutation, weak immigrants, or limited environmental knowledge.Examples include triggered hypermutation, variable local search, dynamic relocation, immigrant schemes, and DNSGA-II; memory methods tend to work better in periodic environments, while older memory methods can converge slowly or lack diversity,,.Source: §2.2 · pp. 6–7
  • Multi-population methods divide search between exploration and exploitation, while prediction approaches estimate future optima or populations, although early prediction may lack historical information.Examples include scout and base populations, multi-population PSO, and clustering; Kalman filtering estimates the next optimum, whereas PPS predicts a center point and manifold but may converge poorly at the beginning.Source: §2.2 · pp. 7–9
  • The authors argue that assuming IID data across times can produce inaccurate prediction models that misdirect the search, despite prior work showing evolutionary optimization can benefit from reused knowledge.Earlier transfer and knowledge-reuse approaches applied evolutionary methods across related image-classification, multiplexer, and vehicle- or arc-routing problems,,.Source: §2.2 · p. 9
  • Because POSs and POFs across times may be correlated without being identical, the authors propose combining transfer learning with EAs to reuse historical information without requiring IID data.The framework builds a population-level prediction model intended for integration with population-based algorithms without extensive modifications.Source: §2.2 · pp. 9–10

3 Transfer Learning based Dynamic Multiobjective Optimization Algorithm

Tr-DMOEA reuses solutions from prior environments to reduce the cost of searching for a dynamic problem’s next Pareto-optimal front. It uses transfer learning to map correlated but non-identical solution distributions into a latent space where they are more similar.

3 Transfer Learning based Dynamic Multiobjective Optimization Algorithm

  • Because solutions from different environments have distinct but correlated distributions, mapping them into a similar latent space enables reuse of past solutions to form an initial population at lower computational cost.This reuse is intended to speed the evolutionary search for the varying Pareto-optimal front.Source: §3 · p. 10, §3.2 · pp. 13–14

3.1 Transfer Component Analysis

  • TCA uses MMD to reduce source–target distribution differences while preserving data variance, learning a low-rank mapping matrix W rather than an entire kernel matrix.The regularized trace optimization is solved by generalized eigenvalue decomposition, with the solution given by the d leading eigenvectors.Source: §3.1 · pp. 11–13

3.2 Tr-DMOEA

  • The method targets a limitation of existing dynamic algorithms, which generally neglect that individuals from different environments are non-independent and identically distributed.The paper presents domain adaptation as a way to address this distribution shift in prediction-based dynamic optimization.Source: §3.2 · p. 14
  • Tr-DMOEA uses domain adaptation to map the current and next environments’ solution distributions into a latent space, then transfers the current POF to initialize the next search.Its Tr-IPG component treats the current POF as the source domain and feasible solutions for the next time as the target domain.Source: §3.2 · pp. 14–15

10 end · 14 end

Tr-IPG uses transfer learning to map the Pareto-optimal front at time t into a latent space and find solutions for an initial population at time t + 1. Its population output can be paired with population-based optimization algorithms.

14 end

  • The TCA takes samples from solutions at times t and t + 1 and outputs transformation matrix W, which is used to construct the latent space.Source: §14 · pp. 15–16
  • Tr-IPG maps the Pareto-optimal front at time t into a latent space, then finds individuals and generates an initial population for the problem at time t + 1.Figure 1 depicts these three steps.Source: §14 · pp. 15–16, Figure 1 · p. 16
  • The numbers of samples used to construct the latent space are predefined, trading potentially better results from more sampling against higher computational cost.The choice depends on available resources.Source: §14 · p. 16
  • To find each new solution, Tr-IPG minimizes its distance in latent space to a mapped solution, solving a single-objective problem with the Interior Point Algorithm.The algorithm seeks x such that ϕ(Ft+1(x)) is closest to l in the Particle in the Latent Space (PLS).Source: §14 · p. 16
  • Because Tr-IPG outputs a population, it can be combined with any population-based optimization algorithm to create a transfer-learning-based dynamic multiobjective evolutionary algorithm.Source: §14 · pp. 16–17
  • TCA's main cost is eigenvalue decomposition, requiring O(d(m1 + m2)^2) time; mapping a POF individual takes O(n1), while the primal-dual interior-point method uses O(m^3L) arithmetic operations.The stated interior-point bound assumes an n-row, m-column constraint matrix with n < m; it has O(√mL) iterations, where L is the total input bits.Source: §14 · p. 17
  • In the FDA4 example, a known POF solution is mapped into a 20-dimensional latent space, and Tr-IPG finds a time t + 1 solution nearest to it for the initial population.FDA4 is described as a three-objective dynamic optimization problem.Source: §14 · p. 17

4 Empirical Study

The study integrates transfer learning with population-based multiobjective algorithms and evaluates the resulting methods on twelve dynamic benchmark functions. It reports stronger DMIGD performance than selected state-of-the-art designs, frequent MIGD improvement over the original algorithms, and mixed robustness results.

4 Empirical Study

  • Tr-DMOEA is designed to work with population-based algorithms, using Tr-IPG to generate an initial population from the previous Pareto front when the environment changes.The case study applies it to NSGA-II, MOPSO, and RM-MEDA as Tr-NSGA-II, Tr-MOPSO, and Tr-RM-MEDA; the original designs are not suited to dynamic optimization.Source: §4 · pp. 17–18

4.1 Performance Metrics, Testing Functions and Settings

  • DMIGD summarizes performance across eight configurations, unlike MIGD’s assessment within one environment; React measures recovery after changes, with MReact and DMReact aggregating React across time points and configurations.IGD measures the difference between the ideal and obtained Pareto fronts; lower IGD and React values are better.Source: §4.1 · pp. 19–20
  • Tests used twelve IEEE CEC 2015 functions across eight configurations, with 20 changes per function and populations of 200; problems varied in dynamics, Pareto-front shapes, and objective counts.FDA4 and FDA5 have three objectives; the other functions have two.Source: §4.1 · pp. 20–21, §4.2.1 · p. 22, Table 2 · p. 22

4.2.1 IGD Metric

  • The authors used common parameters rather than tuning TCA separately for each algorithm or benchmark, noting that individual tuning might improve results but would not isolate the approach’s advantage.They set the TCA expected dimensionality to 20 and μ to 0.5.Source: §4 · p. 18, §4.1 · p. 22, §4.2.1 · p. 23
  • Across 96 total tests each, Tr-MOPSO and Tr-RM-MEDA improved MIGD over MOPSO and RM-MEDA in 70% (67/96) and 73% (70/96) of cases, respectively.Most performance-degradation cases came from FDA5iso and DMOP2iso, whose isolated POFs the authors suspected were poorly handled under inappropriate parameter settings.Source: §4.2.1 · p. 24, Table 4 · p. 24, Table 5 · p. 25
  • In DMIGD benchmark-function experiments, transfer-learning-based algorithms performed much better across different problem characteristics and were much more efficient than the chosen state-of-the-art algorithms, including RND and MOEA/D-KF.Source: §4.2.1 · p. 25

4.2.2 React Metric

  • Tr-NSGA-II and Tr-MOPSO improved DMReact, whereas RM-MEDA robustness appeared to decline, possibly because TCA reduced solution diversity.The authors identify balancing diversity and robustness and selecting parameters as future research topics.Source: §4.2.2 · p. 25

5 Conclusion and Future Works

Tr-DMOEA uses transfer learning to reuse experience across time-varying, non-IID solution distributions and support population-based dynamic multiobjective algorithms. The paper reports improved solution quality and robustness in almost all experiments.

5 Conclusion and Future Works

  • Tr-DMOEA addresses non-IID data: solution distributions differ across times but retain relationships, making classical machine-learning methods difficult to apply.The paper identifies this distribution shift as a reason traditional dynamic optimization algorithms using classical machine learning can struggle to achieve satisfactory performance.Source: §5 · p. 26
  • The framework treats distributions at different times as source and target domains and aligns them in a latent space by minimizing MMD while preserving variance.It uses transferred solutions to construct an initial population for searching at the next time instance.Source: §5 · pp. 26–27
  • On twelve benchmark functions, almost all results showed that transfer learning improved solution quality and algorithm robustness for NSGA-II, MOPSO, and RM-MEDA.The enhanced algorithms were compared with their original designs and selected competing algorithms.Source: §5 · p. 27
  • The authors present this research as an avenue for effective and efficient DMOP evolutionary algorithms, with machine-learning techniques inspiring future work on complex, uncertain real-world applications.The framework is designed to benefit a variety of population-based dynamic multiobjective evolutionary algorithms.Source: §5 · pp. 26–27

6 supplemental Materials

The supplemental materials provide twelve tables of MIGD results for six algorithms across twelve testing functions and eight environments, C1–C8. They define ROC as the ratio of change and use boldface to mark experiments where performance improved.

6 supplemental Materials

  • The twelve tables record MIGD values for NSGA-II, Tr-NSGA-II, MOPSO, Tr-MOPSO, RM-MEDA, and Tr-RM-MEDA across twelve testing functions and environments C1–C8.The tables cover FDA4, FDA5, FDA5iso, FDA5dec, DIMP2, DMOP2, DMOP2iso, DMOP2dec, DMOP3, HE2, HE7, and HE9.Source: §6 · p. 35
  • ROC denotes the ratio of change, and boldface identifies experiments where performance improved.Source: §6 · p. 35
Loading 1612.06093v2…