Source-linked AI summary
A Surrogate-based Approach for Fast Multi-objective Architectural Refactoring Optimization
J. Andrés Diaz-Pace, Daniele Di Pompeo, Antonela Tommasel
TL;DR
Multi-objective architectural optimization is costly because repeatedly invoking analysis solvers for complex models and large search spaces takes substantial computation. This paper integrates regression-based surrogate models into evolutionary refactoring search; it reports about 30% lower evaluation time, alongside reduced variability in explored architectural models.
Problem
Repeated solver evaluations make multi-objective optimization costly for complex architectural models and large search spaces.
Source: §1 · p. 2Method
Results
≈30% lower evaluation time for non-functional properties, with faster convergence, but less variability in explored architectural models and different Pareto-front models than standard evolutionary search.
Source: §1 · p. 2Takeaways & Limitations
The study concludes that surrogate models can provide meaningful efficiency gains in realistic architecture-optimization settings, albeit with less variability on Pareto fronts.
Source: §7 · p. 14The evaluation uses a predefined set of four refactoring actions, fixed sequence length, and relatively small datasets, so surrogate behavior may differ in larger or noisier datasets.
Source: §6 · p. 14
Abstract
from arXiv · showhide
Software model optimization is a process that generates architecture alternatives aimed at improving quantifiable non-functional properties of software systems, such as performance and reliability. Multi-objective evolutionary algorithms are commonly used to explore the search space and help designers identify trade-offs among competing non-functional properties (e.g., through a Pareto front). However, such algorithms face efficiency challenges in complex software models and large design spaces, since evaluating the fitness (i.e., the quality) of each architecture requires analysis tools that become computationally expensive when repeatedly invoked during the search process. In this paper, we explore the construction of surrogate models based on regression techniques to approximate the outputs of these analysis tools at significantly lower computational cost, while maintaining reasonable output accuracy. Our experimental results suggest that surrogate models provide savings of up to $30\%$ in computational time and maintain the Pareto front quality provided by evolutionary algorithms. Also, we observed some differences in the architectural models produced by our approach. Overall, surrogate models constitute a promising approach for scaling multi-objective architecture optimization to larger spaces and complex architectural models.
1 Introduction
This work targets the computational cost of solver-based evaluation in multi-objective architectural optimization. It proposes regression-based surrogate models and evaluates their speed, solution quality, and architectural-model differences on CoCoME.
1 Introduction
- Multi-objective optimization faces costly solver runs, complex architectural models, and large search spaces; time budgets can reduce exploration and affect architectural-alternative quality.Source: §1 · p. 2
- The approach uses regression-based surrogate models trained on architecture features to estimate non-functional properties at lower evaluation cost.Source: §1 · p. 2
- The evaluation uses CoCoME and compares surrogate-assisted results with a state-of-the-art search-based technique while asking about execution time, solution quality, and structural differences.Source: §1 · p. 2
- ≈30% lower evaluation time for non-functional properties, with faster convergence, but less variability in explored architectural models and different Pareto-front models than standard evolutionary search.The reported quality-indicator trade-off is reduced architectural-model variability.Source: §1 · p. 2
2 Related Work
Prior work addresses optimization cost through time budgets and surrogate-assisted search, but architectural surrogates have often been separated from live optimization. This work integrates surrogates into the evolutionary refactoring search.
2 Related Work
- Time-budgeted search can reduce evaluation time, but may produce suboptimal architectural trade-offs depending on the search strategy.Source: §2 · p. 3
- Surrogate-assisted optimization has been studied in evolutionary algorithms and architecture problems, including Bayesian optimization, hidden-constraint modeling, and Pareto-front evaluation under fixed budgets.Source: §2 · p. 3
- Earlier architecture surrogate work predicted performance and modifiability from architectural features, but trained offline on precomputed data and did not guide optimization.Source: §2 · p. 3
- This work trains surrogates incrementally on solver evaluations during search and uses predictions to replace some costly solver calls and guide exploration.Source: §2 · p. 3
3 Approach
The approach integrates surrogate models into EASIER’s genetic-algorithm search, where solver evaluations and predictions assess candidate architectures. It encodes refactoring sequences as fixed-length features and incrementally updates per-objective regression models.
3 Approach
- EASIER applies predefined refactoring actions to architecture candidates and evaluates competing non-functional objectives, such as performance, reliability, energy, and cost, with solvers.Source: §3 · p. 4
- The refactoring catalog includes cloning a node, redeploying a component, moving operation logic, and moving an operation to a new component and node; a feasibility engine validates action sequences.Source: §3 · p. 5
- The surrogate-assisted workflow evaluates a subset of each population with solvers and the remainder with objective-specific surrogate models, using architecture features for predictions.An initial set of surrogates is trained offline and supplied before search.Source: §3 · p. 5, Figure 2 · p. 5
3.1 Feature Encoding of Architectural Models
- Each refactoring sequence becomes a fixed-length numeric feature vector describing its actions and architectural characteristics, with a stable encoding across populations.Source: §3.1 · p. 6
- The encoding combines action unigrams and bigrams with positional action counts, created nodes and components, and device-family and size features.Unigrams count action occurrences; bigrams capture immediately adjacent actions.Source: §3.1 · pp. 6–7, Figure 3 · p. 6
3.2 Incremental Regression Models
- Each objective’s solver-computed values serve as regression targets, with a separate regressor created for each objective.Source: §3.2 · p. 7
- Every k iterations, models can be updated with a fraction of new population data while retaining patterns from earlier instances; gradient boosting can add newly trained estimators.Source: §3.2 · p. 7
4 Evaluation
The evaluation compares surrogate-assisted and standard multi-objective architecture optimization on CoCoME, examining execution time, Pareto-front quality, and the structural characteristics of generated architectures. It tests pre-trained and periodically retrained surrogates using specified GA settings and standard quality indicators.
4 Evaluation
- The evaluation asks whether surrogate assistance reduces execution time while maintaining Pareto-solution quality, and whether it changes the refactoring actions used in explored architectures.Execution time and Pareto quality address RQ1; architecture structure and refactoring actions address RQ2.Source: §4 · p. 8
4.2 Metrics
- Execution time is the computational-cost metric, while Hypervolume and IGD+ assess Pareto-front quality; Figure 4 compares per-iteration times across GA-std and three surrogate strategies.For IGD+, the reference front merges non-dominated solutions across runs; for Hypervolume, the nadir point uses the worst observed value per objective.Source: §4 · p. 8, §4.2 · pp. 9–10, Figure 4 · p. 10
4.1 Experimental Setup
- The setup compares NSGA-II with regression-based surrogate assistance, evaluating half of each iteration’s candidates with solvers and assigning predictions to the remainder.It tests pre-trained surrogates and retraining every 2 or 5 iterations; each configuration runs 31 times with 12 candidates, 102 iterations, and 7 objectives on CoCoME.Source: §4.1 · pp. 8–9
5 Results
Surrogate-assisted optimization reduces execution time while maintaining broadly comparable, indicator-dependent Pareto quality when retraining is frequent. Its search shares many refactoring sequences with the standard GA, yet can produce different architectural trade-offs.
5.1 RQ1: Reductions in overall execution time
- ≈30% lower overall execution time was achieved by the surrogate-assisted GA, with Pareto-front quality competitive when retraining is frequent.The runtime savings came from reducing expensive solver calls.Source: §5.1 · pp. 10–11
- Across 31 runs, GA-std had the best median HV and IGD+, but no pairwise comparison remained significant after Holm correction; SGA-k2 was comparable, especially for IGD+.SGA-k5 showed lower HV and higher IGD+ than the other variants.Source: §5.1 · pp. 10–11
5.2 RQ2: Characterization of architectural models
- The GAs shared a significant portion of refactoring-action sequences, but their main differences appeared in the final action, regardless of the surrogate retraining step.The sequences were compared as a prefix tree after action parameters were omitted.Source: §5.2 · p. 12, Figure 5 · p. 12
- Standard GA favored motn and clone, whereas surrogate-assisted GA favored redo and move, suggesting different structural trade-offs across non-functional objectives.The authors suggest expansion-oriented actions may favor scalability and robustness at higher resource cost, while redistribution-oriented actions may favor energy, power, or price objectives but yield lower performance or reliability gains.Source: §5.2 · p. 12
- The action profiles suggest the GAs explore somewhat different design-space areas, and those architectural differences are difficult to assess from Pareto quality indicators alone.The surrogate-assisted solutions are not necessarily a subset of standard-GA solutions.Source: §5.2 · p. 13, §5.3 · p. 13
5.3 Lessons Learned
- Surrogate predictive performance depends on objective-value series being relatively regular; increasing estimator counts to improve prediction may increase training time.A fixed feature set also kept execution time low because it did not depend on architectural-model size.Source: §5.3 · p. 13
- For CoCoME, unigrams and bigrams over components, nodes, and refactoring actions supported predictions across model sizes and structural variations; TrainTicket evidence remains initial.The authors believe the encoding may extrapolate to other systems based on those initial TrainTicket results.Source: §5.3 · p. 13
- The approach is promising under constrained computational budgets and may produce alternative trade-offs, although further experimentation is needed.The results frame runtime savings and broadly comparable, but potentially variable, Pareto quality as a practical trade-off.Source: §5.3 · p. 13
6 Threats to validity
The evaluation faces threats from feature and sampling choices, model and dataset scope, pipeline and timing variability, and unassessed human factors. These constraints qualify how broadly its findings can be applied.
6 Threats to validity
- Using all encoded features without selection and testing only a half-and-half solver/prediction split could affect surrogate accuracy or training time.The study did not try alternative sampling ratios.Source: §6 · pp. 13–14
- Results may not generalize beyond the evaluated modeling notation, since other languages may support different structures, refactorings, and solvers.Surrogate behavior might vary in another setting.Source: §6 · p. 14
- The search used four refactoring actions and a fixed sequence length, while relatively small datasets leave behavior on larger or noisier industrial-scale data uncertain.Accuracy trends and sampling effects may differ in those settings.Source: §6 · p. 14
- Nondeterministic pipeline behavior and differences in execution environments could affect conclusions and timing measurements despite repeated runs and fixed seeds.Source: §6 · p. 14
- The evaluation did not assess whether architects interpret or trust surrogate predictions and derived models, factors that might influence practical adoption.Source: §6 · p. 14
7 Conclusion
The study finds that interleaving solver evaluations with surrogate predictions can reduce computation time while preserving optimization effectiveness, though Pareto fronts show less variability and generated search spaces differ slightly. It identifies future refinements and provides access to implementation, datasets, and results.
7 Conclusion
- Surrogates integrated with minimal overhead offer meaningful efficiency gains while preserving optimization effectiveness, with less variability on Pareto fronts and slight differences in explored search spaces.The time reductions may let practitioners explore larger design spaces within a given budget; further experiments are needed to confirm the search-space trend and objective-value effects.Source: §7 · p. 14
- Future work includes drift-triggered retraining, alternative model and refactoring encodings, and adaptive solver sampling based on uncertainty, diversity, or predicted reliability.Source: §7 · pp. 14–15
Data Availability.
- The surrogate implementation, datasets, and experimental results are available in the project repository.Source: Data Availability · p. 15