PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimal Scheduling of Isolated Microgrids Using Automated Reinforcement Learning-based Multi-period Forecasting

Yang Li, Ruinong Wang, Zhen Yang

arXiv:2108.06764v1eess.SPcs.LGeess.SY

TL;DR

The paper addresses uncertainty in isolated-microgrid day-ahead scheduling, where accurate multi-period forecasts and error treatment are needed. It proposes PER-AutoRL-based single-step multi-period forecasting, error-distribution revision, and integrated demand-response scheduling. Compared with scheduling without forecasting, the approach reduces operating costs across confidence levels.

Abstract

from arXiv · show

In order to reduce the negative impact of the uncertainty of load and renewable energies outputs on microgrid operation, an optimal scheduling model is proposed for isolated microgrids by using automated reinforcement learning-based multi-period forecasting of renewable power generations and loads. Firstly, a prioritized experience replay automated reinforcement learning (PER-AutoRL) is designed to simplify the deployment of deep reinforcement learning (DRL)-based forecasting model in a customized manner, the single-step multi-period forecasting method based on PER-AutoRL is proposed for the first time to address the error accumulation issue suffered by existing multi-step forecasting methods, then the prediction values obtained by the proposed forecasting method are revised via the error distribution to improve the prediction accuracy; secondly, a scheduling model considering demand response is constructed to minimize the total microgrid operating costs, where the revised forecasting values are used as the dispatch basis, and a spinning reserve chance constraint is set according to the error distribution; finally, by transforming the original scheduling model into a readily solvable mixed integer linear programming via the sequence operation theory (SOT), the transformed model is solved by using CPLEX solver. The simulation results show that compared with the traditional scheduling model without forecasting, this approach manages to significantly reduce the system operating costs by improving the prediction accuracy.

I. INTRODUCTION

The paper addresses uncertainty in isolated-microgrid scheduling by integrating automated reinforcement-learning forecasting with demand-responsive operation. It targets limitations in multi-period accuracy, error modeling, and coordination between forecasting and scheduling.

  • Existing forecasting methods require substantial domain knowledge, have limited generic capability, and often provide inadequate multi-period accuracy for day-ahead scheduling.Source: A. Literature Review · pp. 1–2
  • Isolated microgrids face difficult uncertainty-management problems because their small capacity and lack of main-grid support limit operational flexibility.Source: A. Literature Review · p. 2
  • Prior scheduling approaches commonly use fixed values or rough estimates rather than dynamically integrating sophisticated forecasting models.Source: A. Literature Review · p. 2
  • The paper introduces PER-AutoRL to automate forecasting-model architecture and hyperparameter selection while improving forecasting-to-scheduling integration.Source: B. Contribution of This Paper · p. 2
  • Its forecasting method converts multi-step prediction into multi-period single-step prediction, models error uncertainty, revises predictions, and integrates them into demand-response scheduling.Source: B. Contribution of This Paper · p. 2

B. PER-AutoRL

PER-AutoRL extends automated machine learning to deep reinforcement learning for customized forecasting-model construction. It combines Bayesian architecture and hyperparameter optimization with prioritized experience replay and selectable reward functions.

  • AutoML reduces reliance on expert experience and manual debugging when constructing machine-learning applications.Source: B. PER-AutoRL · p. 3
  • PER-AutoRL uses the Metis sequence-based Bayesian optimizer to select DRL architectures and optimize hyperparameters.Source: B. PER-AutoRL · p. 3
  • The DDPG actor predicts from input data, while the critic evaluates prediction results and supports selection of an optimal prediction policy.Source: B. PER-AutoRL · p. 3
  • The system treats reward functions as tunable hyperparameters and selects the architecture and policy associated with minimal testing MAPE.Source: B. PER-AutoRL · p. 3
  • Prioritized experience replay increases sampling of valuable experiences using TD-error-based ranking, accelerating learning while importance-sampling weights limit oscillation or divergence.Source: B. PER-AutoRL · pp. 3–4

C. Multi-period Forecasting Based on PER-AutoRL

The proposed forecasting method reformulates day-ahead multi-step prediction as multiple single-step series forecasts to avoid error accumulation. It then revises forecasts using a fitted error distribution before scheduling.

2) Revised Forecasting Results:

The method revises load, wind-turbine, and photovoltaic forecasts by adding their expected forecasting errors. These revised values support scheduling, while MAPE and RMSE evaluate forecasting performance.

A. Sequence Operation Theory

Sequence operation theory represents uncertain forecasting errors as probabilistic sequences and combines them through sequence operations. Here, it models WT, PV, load, joint-output, and equivalent-load forecasting errors.

A. Sequence Operation Theory

  • SOT discretizes random-variable distributions into probabilistic sequences and generates new sequences through sequence operations.The sequences encode nonnegative probability values that sum to one.Source: A. Sequence Operation Theory · p. 5
  • WT, PV, and load forecasting errors are modeled as random variables whose distributions are discretized using step length q.The resulting sequences are formed over bounded minimum and maximum error values.Source: B. Sequence Description of Equivalent… · p. 5
  • Addition-type convolution combines WT and PV output-error sequences into a probabilistic sequence for joint-output forecasting errors.The joint-output sequence is subsequently used in equivalent-load error calculations.Source: B. Sequence Description of Equivalent… · p. 5
  • Subtraction-type convolution produces the probabilistic sequence of equivalent-load forecasting errors from load and joint renewable-output errors.The equivalent load is defined as load power minus the combined WT and PV outputs.Source: B. Sequence Description of Equivalent… · p. 5
  • The resulting probabilistic sequence links equivalent-load forecasting-error values with their probabilities for subsequent calculations.The paper uses this representation to obtain the expectation of the equivalent-load forecasting error.Source: B. Sequence Description of Equivalent… · p. 5

IV. OPTIMAL SCHEDULING MODEL OF MICROGRID BASED ON PER-AUTORL FORECASTING

The scheduling model minimizes isolated-microgrid operating costs using revised equivalent-load forecasts, demand response, energy storage, and probabilistic spinning-reserve requirements. Its constraints cover power balance, microturbine operation, storage behavior, and reserve provision.

IV. OPTIMAL SCHEDULING MODEL OF MICROGRID BASED ON PER-AUTORL FORECASTING

  • The objective minimizes total operating costs from microturbine fuel, spinning reserves, startup, and interruptible-load subsidies.Electric demand is divided into fixed and interruptible loads, with subsidies representing user compensation for interruption.Source: A. Optimal Scheduling Model · p. 5, 1) Objective Function · p. 5

MT output constraint

  • Microturbine output is constrained by each unit’s upper and lower power limits.The model also includes microturbine operating-state and startup variables in the scheduling formulation.Source: 1) Objective Function · p. 5, +P CNLOAD · p. 6

Power balance constraint

  • Power balance deploys controllable loads to maintain equilibrium in the isolated microgrid.The balance uses renewable outputs, predicted or revised equivalent load, storage power, and controllable-load terms.Source: 2) Constraint Conditions · p. 5, +P CNLOAD · p. 6

Energy storage system constraints

  • Lead-acid battery constraints govern charge-discharge power, state-of-charge capacity, and initial-to-final energy balance.The storage state transition uses charging and discharging efficiencies with one-hour scheduling periods, and final storage equals initial storage.Source: +P CNLOAD · p. 6

Spinning reserve constraint

  • Spinning reserves supplied by microturbines and the ESS cover deviations between fluctuating equivalent load and its revised prediction.A chance constraint balances reliability against the high cost of preparing for low-probability extreme renewable shortfalls.Source: +P CNLOAD · p. 6

B. Deterministic Transformation of Chance Constraints

The chance-constrained scheduling formulation is converted into mixed integer programming using binary variables and a large positive constant. The resulting model follows the proposed scheduling flowchart and is solved in a readily compatible form.

B. Deterministic Transformation of Chance Constraints

  • The original chance constraint is transformed using the probabilistic sequence of equivalent-load forecasting errors.The probability associated with each discretized error value is incorporated into the deterministic formulation.Source: B. Deterministic Transformation of… · p. 6
  • A very large positive constant replaces an expression incompatible with mixed integer programming, completing the model transformation.After substitution, the scheduling model takes the form of a mixed integer programming problem.Source: B. Deterministic Transformation of… · pp. 6–7
  • Fig. 2 presents the overall flowchart of the proposed scheduling model.The flowchart summarizes the stages leading from forecasting and uncertainty treatment to scheduling-model solution.Source: Figure 2 · p. 7, B. Deterministic Transformation of… · p. 7

V. CASE STUDY

The case study evaluates the approach using a real-world microgrid in North China through numerical simulations implemented in Python on a PC platform.

V. CASE STUDY

  • A real-world North China microgrid is used to examine the proposed approach through numerical simulation.The simulations use Python and run on a PC with two 2.6Hz Intel Core dual-core CPUs and 6 GB RAM.Source: §V · p. 7

A. Introduction of Test System

The test system comprises an isolated microgrid with multiple microturbines, wind and photovoltaic generation, and battery storage. Five years of operating data are divided into 24 time series for forecasting, with an 0.8:0.2 training-testing split.

B. Multi-period Renewable Power Outputs and Load Forecasting

The proposed forecasting approach combines automated reinforcement learning with single-step multi-period prediction and error-distribution revision. Across load, wind, and photovoltaic forecasting, the reported results indicate reduced error accumulation and improved accuracy, providing forecasts for scheduling.

C. Optimal Scheduling Based on PER-AutoRL Forecasting

The forecasting-based scheduling approach improves isolated-microgrid operation by reducing operating costs, spinning-reserve requirements, and calculation effort relative to traditional scheduling without forecasting.

  • The approach uses probability-based inputs and reserve requirements in the traditional no-forecasting comparison, while its scheduling model incorporates forecasting results.The scheduling model is evaluated against this traditional formulation and against a PSO–MCS chance-constraint solution.Source: C. Optimal Scheduling Based on… · p. 9, D. Performance Comparison · p. 10
  • At the 95% confidence level, the forecasting-based schedule mainly uses MT3 and ESS, starts MT1 only at 24:00, and does not require MT2.Without forecasting, MT1 and MT3 provide reserve in most periods, while MT2 joins when load demand increases.Source: C. Optimal Scheduling Based on… · p. 9
  • Forecast accuracy is closely related to operating cost, with greater prediction errors associated with higher microgrid operating costs.The paper reports that prediction methods therefore have a significant impact on operating costs.Source: C. Optimal Scheduling Based on… · p. 9, Figure 10 · p. 9
  • The forecasting-based schedule requires less spinning reserve than scheduling without forecasting in 66.67% of periods at the 95% confidence level.The comparison is reported for the required spinning-reserve capacity.Source: C. Optimal Scheduling Based on… · p. 9, Figure 11 · p. 9
  • 18.3%, 36.7%, and 38.4% operating-cost reductions are reported at the 99%, 95%, and 90% confidence levels, respectively.Higher confidence levels increase spinning-reserve costs, creating a reliability–economy trade-off.Source: C. Optimal Scheduling Based on… · p. 9
  • The proposed scheduling approach reduces operating costs and achieves faster calculation than traditional scheduling without forecasting.The authors attribute the economic improvement to integrating PER-AutoRL forecasting into scheduling.Source: §VI · p. 10, D. Performance Comparison · p. 10
Loading 2108.06764v1…