PaperlayerView exact revision on arXiv

Source-linked AI summary

REMBGO for Evaluation-Limited Optimization: CEC2022 Benchmarking and EAF-STATCOM Generalization

Vikram Chekuri, Narapareddy Ramarao, Prashanth N A

arXiv:2610.13471v1cs.NE

TL;DR

Expensive simulation-based tuning makes optimizer performance near the application’s evaluation budget important, while large-budget rankings may not carry over. The paper refines EMBGO as REMBGO and evaluates its parent-relative performance and gain transfer. Results show budget-dependent comparisons, with REMBGO’s advantage concentrated in restricted-budget conditions rather than universal superiority.

  • Problem

    Expensive EAF–STATCOM simulations constrain tuning to 1040 objective evaluations, and rankings from large-budget benchmarks may not represent performance at that application-scale budget.

    Source: §1 · p. 1

  • Method

    REMBGO adds elitism, a fitness-weighted centroid, normalized neighbourhood control, perturbation, stagnation handling, and partial restart to EMBGO, then tests frozen gains on unseen stochastic arcs.

    Source: §1 · pp. 1–2

  • Results

    At 1040 evaluations, REMBGO’s EMBGO win/tie/loss counts are 4/6/2 at D=10 and 9/1/2 at D=20, with its strongest parent advantage at smaller budgets.

    Source: §6 · p. 10

  • Takeaways & Limitations

    Optimizer selection for simulation-expensive tuning should use a representative budget; REMBGO is not universally best across budgets and dimensions.

    Source: §8 · p. 12

    Evidence covers CEC2022 at D=10 and D=20 and one simulated EAF–STATCOM plant family, rather than hardware testing.

    Source: §7 · p. 11

Abstract

from arXiv · show

Metaheuristic comparisons are often reported at evaluation budgets far larger than an expensive engineering simulation can afford. This paper presents REMBGO, an evaluation-budget-aware refinement of EMBGO. Its contribution is not any single operator: it is the integration of population preservation, scale-normalized exploration and recovery mechanisms whose parent-relative benefit is concentrated under restricted function-evaluation budgets. REMBGO is evaluated on CEC2022 at a primary, application-aligned budget of 1040 function evaluations, with a surrounding budget ladder reported descriptively to characterise budget dependence rather than as a family of controlled hypotheses. A component-removal study finds no individual mechanism whose removal survives multiplicity correction, consistent with an integrated rather than a single-operator contribution. The algorithm is then applied to EAF--STATCOM controller tuning, and the resulting frozen gain vectors are transferred without retuning to twenty pre-specified unseen stochastic arc realizations ($210$ fixed-gain simulations, all feasible). The selected gain-vector main effect accounts for only $3.6$--$5.8\%$ of the crossed validation variation, against approximately $61.4\%$ for interaction plus residual, and the matched REMBGO--EMBGO paired differences are controller-dependent in sign and not statistically significant. REMBGO is therefore presented as a resource-aware refinement whose benefit must be stated together with the evaluation budget, not as a universally superior optimizer.

1 Introduction

Because EAF–STATCOM tuning is limited to 1040 objective evaluations, this study asks whether optimizer rankings and REMBGO’s parent-relative benefits hold at application-scale budgets. It also tests whether selected controller gains retain performance on unseen stochastic EAF trajectories.

1 Introduction

  • REMBGO extends EMBGO with elitism, a fitness-weighted centroid, normalized neighbourhood control, perturbations, stagnation handling, and partial restart.It retains the battle-game core while targeting restricted-evaluation search.Source: §1 · p. 1
  • Prior budget-dependent benchmarking includes COCO and its extension, while cited STATCOM and EAF studies use fixed experimental settings.The paper addresses budget-resolved comparison and transfer questions not covered by those application studies.Source: §1 · pp. 1–2
  • The methodological gap is a matched evaluation of REMBGO against EMBGO across budgets, at the exact application budget, and on unseen stochastic trajectories.A component-removal analysis accompanies the parent comparison.Source: §1 · p. 2
  • At 1040 evaluations, the study compares five optimizers across independently configured CEC2022 budgets, including the exact application budget.The benchmark uses D=10 and D=20 with thirty runs per function.Source: §1 · p. 2
  • The transfer study replays one selected gain vector per optimizer and controller across twenty unseen stochastic arc realizations.The study comprises 210 fixed-gain EAF–STATCOM simulations.Source: §1 · p. 2

2 Application Context and Objective Function

The application is an EAF–STATCOM EMT simulation with two six-parameter controllers, tuned under a fixed 1040-evaluation budget. Its scalar objective combines disturbance performance with quality and safety penalties.

2 Application Context and Objective Function

  • The simulated plant is an 11 kV, 50 Hz EAF–STATCOM system, and the study tunes six parameters for each of the DQ-PI and Lyapunov controllers.The DQ-PI uses decoupled current PI control; the Lyapunov design uses nonlinear inner current feedback.Source: §2 · p. 2
  • Each tuning run uses 40 candidates and 25 update iterations, totaling 1040 objective evaluations including initialization.The arc-noise stream is held at one stochastic realization during tuning, so repeated seeds measure optimizer stochasticity rather than trajectory variation.Source: §2 · p. 2
  • A complete 1040-evaluation optimization seed took approximately 98–273 minutes, depending on controller, worker count, and computing platform.These runtimes establish simulation cost and are not used to compare optimizer execution speed.Source: §2 · p. 2
  • The objective scores an EMT simulation over 0.50–1.00 s using reactive-power fluctuation and one-cycle-RMS PCC-voltage features normalized to a common reference.The score is a short-window optimization objective, not an IEC flicker index.Source: §2 · pp. 2–3
  • The scalar objective adds graded reactive-power and dc-link penalties plus hard feasibility gates and soft safety costs.A candidate is hard-feasible when no hard gate is triggered; Jtot is the sole objective in tuning and transfer.Source: §2 · pp. 2–3

3 REMBGO Formulation

REMBGO retains EMBGO’s battle-game core but adds population preservation, guidance, exploration, and recovery mechanisms within strict evaluation-budget accounting. Its search cycle ranks and guides candidates, generates and selects trials, then partially restarts weak candidates when stagnation or low diversity warrants recovery.

3.1 Population model and notation

  • The population is ranked by fitness, with the best and worst candidates identified; diversity is measured in normalized coordinates.Normalized coordinates are defined by subtracting the lower bound and dividing by the variable range.Source: §3.1 · p. 3

3.2 Inherited EMBGO structure and its limitations

  • EMBGO’s raw-unit neighbourhood radius can be dominated by the widest-ranging variable, while its mean attractor can be pulled toward poor candidates.It also lacks explicit elite preservation and diversity recovery.Source: §3.2 · p. 3

3.3 REMBGO search cycle

  • REMBGO runs a fixed sequence of ranking and elite preservation, guidance, trial generation, budgeted evaluation, greedy survivor selection, and conditional partial restart.The implementation described is the one used in all reported experiments.Source: §3.3 · p. 3, §3 · p. 3

Population guidance and preservation

  • REMBGO preserves the three best players and forms a fitness-weighted centroid from eligible candidates; its neighbourhood radius uses normalized coordinates and shrinks over the run.Candidates above the fixed fitness eligibility threshold of 107 are excluded from the centroid; poorer eligible candidates receive smaller weights.Source: Population guidance and preservation · pp. 3–4

Trial-generation portfolio

  • Sequential operator tests select opposition learning, best-centred Cauchy mutation, a radius-gated core, or opponent battle; the step scale decays over the run.The core uses a triple-leader blend for some candidates within the radius, while candidates outside it use Lévy exploration.Source: Trial-generation portfolio · p. 4
  • In opponent battle, a player recombines dimension-wise with a strictly better opponent; otherwise it takes a cosine-modulated step.The opponent is drawn uniformly from the other players.Source: Trial-generation portfolio · p. 5

Recovery and survivor handling

  • Stagnation-dependent jitter perturbs trials, while reflective repair handles boundary violations and partial restart regenerates the weakest 30% of players.At population size 40, the restart comprises 12 players split 6/4/2 among three constructions.Source: Recovery and survivor handling · p. 5

3.4 Evaluation accounting

  • With three elites copied but all 40 trial rows evaluated, each generation spends 40 evaluations for 37 new candidates; partial final generations meet the requested budget exactly.The three copied elite rows account for 7.5% of generation-level calls, without compensation.Source: §3.4 · pp. 5–6

4 Experimental Design and Statistical Methods

The experimental design uses matched, repeated optimizer comparisons under strict evaluation budgets, with separate configurations for each budget. Statistical tests operate on matched function-level outcomes, and archived records support regeneration of reported results.

4 Experimental Design and Statistical Methods

  • The comparison follows guidance for bio-inspired optimization studies, using matched budgets, shared initialization, repeated runs, and nonparametric paired testing.The comparison methodology also applies Holm correction within each planned function family.Source: §4 · p. 6

4.1 Algorithms

  • Five implementations share a strict budget controller and per-evaluation best-so-far recording; FP-SHADE uses fixed population rather than population reduction.Its results therefore do not represent population-reducing SHADE variants.Source: §4.1 · p. 6

4.2 Benchmark and evaluation budgets

  • The CEC2022 comparison covers twelve functions at D=10 and D=20, with thirty official-seed runs and identical initial populations for each matched condition.Each of fourteen budgets per dimension is run as a separate experiment, and 1040 evaluations is the exact application-aligned rung.Source: §4.2 · pp. 6–7

4.3 Statistical methods

  • Per-function median terminal errors determine ranks; matched pairs use two-sided exact Wilcoxon signed-rank tests with Holm correction across twelve functions within each budget.Budgets are not compared inferentially against one another because cross-rung multiplicity is uncontrolled.Source: §4.3 · p. 7

4.4 Reproducibility

  • Benchmark records are archived with optimizer, dimension, function, seed, and budget metadata, allowing tables and figures to be regenerated without rerunning optimizers.Replaying the ten tuning-arc cases reproduced stored Jtot values within a maximum absolute difference of 2.03×10^-8.Source: §4.4 · p. 7

5 Results

REMBGO’s parent-relative advantage is concentrated at restricted evaluation budgets, while optimizer rankings change across budgets. At the 1040-evaluation budget, no individual mechanism removal is significant, and fixed-gain transfer shows substantial arc-by-vector interaction and no statistically significant matched parent advantage.

5.1 Optimizer ranking at different evaluation budgets

  • At the smallest budgets PSO and GWO lead at D=10 with 200 evaluations, while GWO leads at D=20 with 125 evaluations; FP-SHADE leads at the largest budgets.At D=10 and 200 evaluations, PSO and GWO share mean rank 1.917; at D=20 and 125 evaluations, GWO has mean rank 1.250. FP-SHADE's lowest observed mean ranks at the largest budgets are 1.625 (D=10) and 1.542 (D=20).Source: §5.1 · p. 7, Figure 1 · p. 8, Table 2 · p. 8
  • At 1040 evaluations, REMBGO has the lowest observed D=10 mean rank (2.250; omnibus p = 0.089), while GWO leads D=20 (2.250; REMBGO 2.583).The omnibus test rejects equal ranks at D=20 (p = 0.022), but not at D=10.Source: §5.1 · p. 7, Figure 1 · p. 8, Table 2 · p. 8

5.2 REMBGO relative to its parent

  • At 1040 evaluations, Holm-corrected REMBGO/EMBGO records are 4/6/2 at D=10 and 9/1/2 at D=20; the advantage is strongest when evaluations are scarce and narrows or reverses at large budgets.At the smallest budgets the records are 10/2/0 at D=10 and 11/1/0 at D=20; at the largest budgets they are 0/9/3 and 0/8/4, respectively.Source: §5.2 · p. 8, Figure 2 · p. 9

5.3 Component behaviour at the application budget

  • No mechanism removal is significant after suite-level Holm correction; descriptive penalties differ by dimension, with greedy selection most visible at D=10 and Cauchy mutation and the triple-leader move at D=20.The largest D=10 penalty is +0.054 for greedy survivor selection; at D=20, Cauchy mutation is +0.052 and the triple-leader move is +0.042.Source: §5.3 · p. 8, Table 3 · p. 9

5.4 Fixed-gain EAF–STATCOM generalization

  • Selected-vector main effects account for 3.64% (DQ-PI) and 5.75% (Lyapunov) of validation variation, versus 61.38% and 61.46% for arc-by-vector interaction and residual.The ten selected vectors were evaluated on identical pre-specified unseen arcs; all 210 fixed-gain simulations were feasible.Source: §5.4 · pp. 9–10
  • Across the ten selected vectors, the mean shift from STO-SR training to the twenty-unseen-arc mean is +0.0198, but its direction and size depend on the controller.DQ-PI vectors shift by +0.0059 on average, while Lyapunov vectors shift by +0.0338; Lyapunov retains lower absolute Jtot.Source: §5.4 · p. 10
  • Matched REMBGO–EMBGO differences on unseen arcs are controller-dependent in sign and not statistically significant; neither controller family establishes a generalization advantage.For DQ-PI, the median difference is +0.001871 (signed-rank p = 0.498); for Lyapunov, it is −0.006007 (p = 0.165). Both reported intervals include zero.Source: §5.4 · p. 10

6 Discussion

REMBGO’s parent-relative benchmark advantage is concentrated at restricted evaluation budgets, while the fixed-gain transfer results show that stochastic arc and interaction effects outweigh the selected-vector effect. The discussion also notes that an objective-scale-dependent implementation cutoff was tested at the primary budget.

6 Discussion

  • At 1040 evaluations, REMBGO improves on EMBGO on 4/6/2 functions at D=10 and 9/1/2 at D=20; its advantage is strongest at the smallest budgets and narrows or reverses as budgets grow.The budget ladder also places swarm methods ahead at the smallest budgets and FP-SHADE ahead once many evaluations are available.Source: §6 · p. 10
  • Figure 4 shows fixed-gain validation distributions for DQ-PI and Lyapunov over twenty unseen corrected arc realizations, plus a partition of crossed variation.The corrected base-42 reference case is excluded.Source: Figure 4 · p. 11
  • Median per-function differences in median log10 terminal error favor REMBGO by −0.06 and −0.11 decades at D=10 and D=20, respectively, with lower median error on nine of twelve functions at each dimension.These are descriptive suite-level magnitudes, not replacements for Holm-corrected function-wise inference.Source: §6 · pp. 10–11
  • The retained absolute centroid-eligibility cutoff fi < 107 is objective-scale dependent and activates on a subset of CEC2022 functions, but removing it showed no Holm-significant terminal-performance change at 1040 evaluations.The audit found activation confined to F1, F6, and F8.Source: §6 · p. 11
  • Selected-vector main effects account for 3.6% of DQ-PI and 5.8% of Lyapunov validation variation, versus about 61.4% for interaction plus residual; no component removal survives multiplicity correction.The crossed transfer test uses five selected gain vectors per controller over twenty unseen stochastic arcs, with no re-optimization or optimizer ranking.Source: §6 · p. 11, Figure 4 · p. 11

7 Limitations

The paper limits broad optimizer-ranking claims because its benchmark suite is small and its application validation is simulation-based and restricted in scope. It also notes that ranking and tuning choices can change which optimizer appears best.

7 Limitations

  • The benchmark covers CEC2022 at D=10 and D=20 with thirty runs per condition, and within-budget Holm correction leaves win/tie/loss counts describing breadth and direction rather than magnitude.There is no cross-budget multiplicity control.Source: §7 · p. 11
  • CEC2022’s twelve functions per dimension are limited benchmark breadth, so cross-suite validation remains necessary before making broader optimizer-ranking claims.Thirty repetitions, two dimensions, multiple budgets, a matched parent comparison, component removals, and an engineering transfer test partly compensate for this scope.Source: §7 · p. 11
  • Published analyses indicate that ranking method and the tuning effort applied to competing algorithms can change which optimizer appears best on this suite.Source: §7 · p. 12

8 Conclusion

The conclusion presents REMBGO as a budget-dependent refinement rather than a universally best optimizer. In the EAF–STATCOM transfer study, frozen gain vectors remained feasible across unseen arcs, but the comparison with EMBGO did not establish application-side superiority.

8 Conclusion

  • Optimizer ordering depends on evaluation budget: REMBGO is competitive at the application budget, while GWO leads at D=20 and FP-SHADE leads at large budgets.At 1040 evaluations REMBGO holds the lowest observed mean rank at D=10.Source: §8 · p. 12
  • All ten frozen champions remained feasible across twenty pre-specified unseen stochastic arcs, but the selected-vector effect was much smaller than arc realization and its interaction with the vector.The average shift from the training objective to the unseen-arc mean was +0.0198.Source: §8 · p. 12
  • Matched REMBGO−EMBGO paired differences were controller-dependent in sign and not statistically significant for either architecture, supporting fixed-gain transfer robustness rather than an application-side superiority claim.The conclusion points to multi-trajectory or risk-aware tuning under the same restricted-budget setting as future work.Source: §8 · p. 12

Supplementary Material

The supplement collects secondary material on the budget ladder, sensitivity and component diagnostics, fixed-gain generalization, and reproducibility. Its tables and figures regenerate from archived result files without rerunning optimizers.

Supplementary Material

  • Supplementary Section S1 gives the complete evaluation-budget ladder, and Section S2 gives the sensitivity study at 1040 evaluations.Source: Supplementary Material · p. 14
  • Supplementary Sections S3 and S4 provide additional component diagnostics and the full fixed-gain EAF–STATCOM generalization results.Source: Supplementary Material · p. 14
  • Supplementary Section S5 gives reproducibility and statistical conventions plus an application-objective summary; all tables and figures regenerate from archived results without rerunning an optimizer.Source: Supplementary Material · p. 14

S1 Complete five-optimizer evaluation-budget ladder

Table S1 reports the CEC2022 budget ladder’s optimizer ranks and REMBGO–EMBGO comparisons, with inference controlled within each rung rather than across rungs.

S1 Complete five-optimizer evaluation-budget ladder

  • Cross-rung multiplicity is not controlled, so the ladder’s rungs are not compared inferentially with one another.Holm correction is applied across the twelve functions within a single rung.Source: S1 Complete five-optimizer… · p. 14
  • Table S1 lists each optimizer’s mean Friedman rank, the lowest-mean-rank optimizer, the per-rung omnibus Friedman p-value, and the Holm-corrected REMBGO–EMBGO win/tie/loss record.The record covers twelve CEC2022 functions at each budget rung.Source: Table S1 · p. 14

S2 Sensitivity at the exact 1040-evaluation budget

The supplementary sections report sensitivity and mechanism diagnostics, characterize fixed-gain transfer and objective conventions, and audit a finite centroid-eligibility cutoff.

S2 Sensitivity at the exact 1040-evaluation budget

S3 Additional REMBGO component diagnostics

  • On symmetric CEC2022 bounds, the restart reference is the origin, so its multiplicative-neighbour rule injects identical zero vectors; benchmark results remain conditional on that implementation.The alternative-reference diagnostic detected no change on the isotropic benchmark, which is not evidence of equality.Source: S3 Additional REMBGO component… · p. 16
  • On the isotropic CEC2022 box, runs with and without range normalization were bit-identical for every tested seed, function, and budget; only temporal decay was active.Range normalization is retained because it becomes active on anisotropic domains outside the CEC2022 specification.Source: S3 Additional REMBGO component… · p. 16

S4 Fixed-gain EAF–STATCOM generalization: full results

  • Tables S4–S6 report fixed-vector replay performance, matched REMBGO–EMBGO differences, and crossed validation variation, respectively; these are descriptive of post-selected champions, not optimizer comparisons.Each Table S4 row replays one selected vector across twenty matched unseen trajectories.Source: S4 Fixed-gain EAF–STATCOM… · p. 16, Table S4 · p. 17, Table S5 · p. 17

S5 Reproducibility, statistical conventions and compact objective summary

  • The application objective combines normalized reactive-power and voltage performance with graded power and dc-link penalties and safety costs; component removals measure changes within an interacting implementation.With a fixed call budget, disabling a branch redistributes search exposure, while removing elitism also changes the number of new candidates per generation.Source: S5 Reproducibility, statistical… · p. 18
  • All 210 fixed-gain replay cases were hard-feasible; DQ-PI cases incurred nonzero dc-link quality penalties, while every Lyapunov case met the zero-penalty screen.DQ-PI peak deviation was 5.675–5.979%, above the 3% free-quality band but below the 8% hard gate.Source: S5 Reproducibility, statistical… · p. 18

S6 Threshold audit and finite-cutoff sensitivity

  • The finite cutoff is an inherited absolute centroid-eligibility threshold, and its audit recorded 67,661 of 707,724 finite candidate observations as eligible; a rank- or quantile-based rule is only a possible future refinement.The proposed variant is a post-campaign sensitivity test, not a replacement optimizer campaign; frozen benchmark results are unchanged.Source: S6 Threshold audit and finite-cutoff… · pp. 18–19
  • Across 180 cutoff-active runs, the no-cutoff variant was better, tied, or worse in 82/4/94 seed pairs; for F6 at D=10, its paired shift was +0.044 decades despite lower separate medians.The separate medians were 3.22 × 10^4 and 1.65 × 10^4, but the matched-seed paired shift is the quantity used for inference.Source: S6 Threshold audit and finite-cutoff… · p. 19
Loading 2610.13471v1…