PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimal Scheduling of Isolated Microgrids Using Automated Reinforcement Learning-based Multi-period Forecasting

Yang Li, Ruinong Wang, Zhen Yang

arXiv:2108.06764v1eess.SPcs.LGeess.SY

TL;DR

The paper addresses uncertainty in isolated-microgrid day-ahead scheduling, where accurate multi-period forecasts and error treatment are needed. It proposes PER-AutoRL-based single-step multi-period forecasting, error-distribution revision, and integrated demand-response scheduling. Compared with scheduling without forecasting, the approach reduces operating costs across confidence levels.

  • Problem

    Existing forecasting models often provide insufficient multi-period accuracy, inadequately model forecasting-error uncertainty, and are weakly integrated with microgrid scheduling.

  • Method

    The paper combines PER-AutoRL single-step multi-period forecasting, TLS-based forecast revision, demand-response scheduling, and chance-constrained reserve planning.

  • Results

    Operating costs versus scheduling without forecasting are reduced by 18.3% at 99%, 36.7% at 95%, and 38.4% at 90% confidence.

  • Takeaways & Limitations

    Integrating the proposed forecasting method with scheduling improves forecasting accuracy and operational economy for the studied isolated microgrid.

Abstract

from arXiv · show

In order to reduce the negative impact of the uncertainty of load and renewable energies outputs on microgrid operation, an optimal scheduling model is proposed for isolated microgrids by using automated reinforcement learning-based multi-period forecasting of renewable power generations and loads. Firstly, a prioritized experience replay automated reinforcement learning (PER-AutoRL) is designed to simplify the deployment of deep reinforcement learning (DRL)-based forecasting model in a customized manner, the single-step multi-period forecasting method based on PER-AutoRL is proposed for the first time to address the error accumulation issue suffered by existing multi-step forecasting methods, then the prediction values obtained by the proposed forecasting method are revised via the error distribution to improve the prediction accuracy; secondly, a scheduling model considering demand response is constructed to minimize the total microgrid operating costs, where the revised forecasting values are used as the dispatch basis, and a spinning reserve chance constraint is set according to the error distribution; finally, by transforming the original scheduling model into a readily solvable mixed integer linear programming via the sequence operation theory (SOT), the transformed model is solved by using CPLEX solver. The simulation results show that compared with the traditional scheduling model without forecasting, this approach manages to significantly reduce the system operating costs by improving the prediction accuracy.

I. INTRODUCTION

The paper addresses uncertainty in isolated-microgrid scheduling by integrating automated reinforcement-learning forecasting with demand-responsive operation. It targets limitations in multi-period accuracy, error modeling, and coordination between forecasting and scheduling.

  • Existing forecasting methods require substantial domain knowledge, have limited generic capability, and often provide inadequate multi-period accuracy for day-ahead scheduling.
  • Isolated microgrids face difficult uncertainty-management problems because their small capacity and lack of main-grid support limit operational flexibility.
  • Prior scheduling approaches commonly use fixed values or rough estimates rather than dynamically integrating sophisticated forecasting models.
  • The paper introduces PER-AutoRL to automate forecasting-model architecture and hyperparameter selection while improving forecasting-to-scheduling integration.
  • Its forecasting method converts multi-step prediction into multi-period single-step prediction, models error uncertainty, revises predictions, and integrates them into demand-response scheduling.

B. PER-AutoRL

PER-AutoRL extends automated machine learning to deep reinforcement learning for customized forecasting-model construction. It combines Bayesian architecture and hyperparameter optimization with prioritized experience replay and selectable reward functions.

  • AutoML reduces reliance on expert experience and manual debugging when constructing machine-learning applications.
  • PER-AutoRL uses the Metis sequence-based Bayesian optimizer to select DRL architectures and optimize hyperparameters.
  • The DDPG actor predicts from input data, while the critic evaluates prediction results and supports selection of an optimal prediction policy.
  • The system treats reward functions as tunable hyperparameters and selects the architecture and policy associated with minimal testing MAPE.
  • Prioritized experience replay increases sampling of valuable experiences using TD-error-based ranking, accelerating learning while importance-sampling weights limit oscillation or divergence.

C. Multi-period Forecasting Based on PER-AutoRL

The proposed forecasting method reformulates day-ahead multi-step prediction as multiple single-step series forecasts to avoid error accumulation. It then revises forecasts using a fitted error distribution before scheduling.

  • Traditional multi-step forecasting feeds each prediction into the next step, causing inevitable error accumulation and declining accuracy.
  • The proposed PER-AutoRL method transforms multi-step forecasting into multi-period single-step forecasting.
  • The dataset is reconstructed into 24 one-hour time series, with separate PER-AutoRL architectures and hyperparameters automatically determined for each series.
  • Forecasts for each period are integrated into a 24-hour day-ahead profile, then revised using expected errors estimated from PER-AutoRL error distributions.
  • The TLS distribution describes PER-AutoRL prediction errors using mean, standard deviation, and shape-coefficient parameters.
  • For WT forecasting errors, TLS fits the probability distribution better than normal and logistic distributions over the stated historical period.

2) Revised Forecasting Results:

The method revises load, wind-turbine, and photovoltaic forecasts by adding their expected forecasting errors. These revised values support scheduling, while MAPE and RMSE evaluate forecasting performance.

  • Expected forecasting errors are obtained from the prediction-error distribution and used to modify the initial forecasts.
  • The revised forecasts cover load, WT output, and PV output, while the underlying forecasts and their errors are distinguished separately.
  • MAPE and RMSE are used as the common indicators for evaluating forecasting methods.

A. Sequence Operation Theory

Sequence operation theory represents uncertain forecasting errors as probabilistic sequences and combines them through sequence operations. Here, it models WT, PV, load, joint-output, and equivalent-load forecasting errors.

  • A. Sequence Operation Theory: SOT discretizes random-variable distributions into probabilistic sequences and generates new sequences through sequence operations.The sequences encode nonnegative probability values that sum to one.
  • A. Sequence Operation Theory: WT, PV, and load forecasting errors are modeled as random variables whose distributions are discretized using step length q.The resulting sequences are formed over bounded minimum and maximum error values.
  • A. Sequence Operation Theory: Addition-type convolution combines WT and PV output-error sequences into a probabilistic sequence for joint-output forecasting errors.The joint-output sequence is subsequently used in equivalent-load error calculations.
  • A. Sequence Operation Theory: Subtraction-type convolution produces the probabilistic sequence of equivalent-load forecasting errors from load and joint renewable-output errors.The equivalent load is defined as load power minus the combined WT and PV outputs.
  • A. Sequence Operation Theory: The resulting probabilistic sequence links equivalent-load forecasting-error values with their probabilities for subsequent calculations.The paper uses this representation to obtain the expectation of the equivalent-load forecasting error.

IV. OPTIMAL SCHEDULING MODEL OF MICROGRID BASED ON PER-AUTORL FORECASTING

The scheduling model minimizes isolated-microgrid operating costs using revised equivalent-load forecasts, demand response, energy storage, and probabilistic spinning-reserve requirements. Its constraints cover power balance, microturbine operation, storage behavior, and reserve provision.

  • IV. OPTIMAL SCHEDULING MODEL OF MICROGRID BASED ON PER-AUTORL FORECASTING: The objective minimizes total operating costs from microturbine fuel, spinning reserves, startup, and interruptible-load subsidies.Electric demand is divided into fixed and interruptible loads, with subsidies representing user compensation for interruption.
  • MT output constraint: Microturbine output is constrained by each unit’s upper and lower power limits.The model also includes microturbine operating-state and startup variables in the scheduling formulation.
  • Power balance constraint: Power balance deploys controllable loads to maintain equilibrium in the isolated microgrid.The balance uses renewable outputs, predicted or revised equivalent load, storage power, and controllable-load terms.
  • Energy storage system constraints: Lead-acid battery constraints govern charge-discharge power, state-of-charge capacity, and initial-to-final energy balance.The storage state transition uses charging and discharging efficiencies with one-hour scheduling periods, and final storage equals initial storage.
  • Spinning reserve constraint: Spinning reserves supplied by microturbines and the ESS cover deviations between fluctuating equivalent load and its revised prediction.A chance constraint balances reliability against the high cost of preparing for low-probability extreme renewable shortfalls.

B. Deterministic Transformation of Chance Constraints

The chance-constrained scheduling formulation is converted into mixed integer programming using binary variables and a large positive constant. The resulting model follows the proposed scheduling flowchart and is solved in a readily compatible form.

  • B. Deterministic Transformation of Chance Constraints: The original chance constraint is transformed using the probabilistic sequence of equivalent-load forecasting errors.The probability associated with each discretized error value is incorporated into the deterministic formulation.
  • B. Deterministic Transformation of Chance Constraints: A very large positive constant replaces an expression incompatible with mixed integer programming, completing the model transformation.After substitution, the scheduling model takes the form of a mixed integer programming problem.
  • B. Deterministic Transformation of Chance Constraints: Fig. 2 presents the overall flowchart of the proposed scheduling model.The flowchart summarizes the stages leading from forecasting and uncertainty treatment to scheduling-model solution.

V. CASE STUDY

The case study evaluates the approach using a real-world microgrid in North China through numerical simulations implemented in Python on a PC platform.

  • V. CASE STUDY: A real-world North China microgrid is used to examine the proposed approach through numerical simulation.The simulations use Python and run on a PC with two 2.6Hz Intel Core dual-core CPUs and 6 GB RAM.

A. Introduction of Test System

The test system comprises an isolated microgrid with multiple microturbines, wind and photovoltaic generation, and battery storage. Five years of operating data are divided into 24 time series for forecasting, with an 0.8:0.2 training-testing split.

  • The microgrid contains 3 microturbine units, 1 wind-turbine unit, photovoltaic panels, and a battery pack.
  • Five years of wind, photovoltaic, and load data from January 1, 2015, to December 31, 2019, are used to verify the forecasting method.
  • The datasets are divided into 24 new time series and split 0.8:0.2 for model training and testing.
  • The scheduling feasibility study uses this microgrid system with specified microturbine parameters.

B. Multi-period Renewable Power Outputs and Load Forecasting

The proposed forecasting approach combines automated reinforcement learning with single-step multi-period prediction and error-distribution revision. Across load, wind, and photovoltaic forecasting, the reported results indicate reduced error accumulation and improved accuracy, providing forecasts for scheduling.

  • PER-AutoRL automatically selects forecasting model architectures and hyperparameters from multi-sub time series of wind, photovoltaic, and load data.For load modeling, only three hyperparameter sets produced MAPEs above 15%, and suitable settings required few optimization iterations.
  • The proposed method maintains forecasting errors in a smaller, stable interval over 72 hours, unlike multi-step forecasting with accumulating errors.
  • Across different prediction models, the proposed forecasting method produces results closer to the ideal fitting line than multi-step forecasting.
  • The one-day load forecasts appropriately reflect the observed daily pattern of increases, breaks, and decreases.
  • The proposed method fits the real wind-turbine and photovoltaic generation curves suitably.
  • Error-distribution revision further improves accuracy over single-step prediction, while wind and photovoltaic outputs have larger errors than load.

C. Optimal Scheduling Based on PER-AutoRL Forecasting

The forecasting-based scheduling approach improves isolated-microgrid operation by reducing operating costs, spinning-reserve requirements, and calculation effort relative to traditional scheduling without forecasting.

  • The approach uses probability-based inputs and reserve requirements in the traditional no-forecasting comparison, while its scheduling model incorporates forecasting results.The scheduling model is evaluated against this traditional formulation and against a PSO–MCS chance-constraint solution.
  • At the 95% confidence level, the forecasting-based schedule mainly uses MT3 and ESS, starts MT1 only at 24:00, and does not require MT2.Without forecasting, MT1 and MT3 provide reserve in most periods, while MT2 joins when load demand increases.
  • Forecast accuracy is closely related to operating cost, with greater prediction errors associated with higher microgrid operating costs.The paper reports that prediction methods therefore have a significant impact on operating costs.
  • The forecasting-based schedule requires less spinning reserve than scheduling without forecasting in 66.67% of periods at the 95% confidence level.The comparison is reported for the required spinning-reserve capacity.
  • 18.3%, 36.7%, and 38.4% operating-cost reductions are reported at the 99%, 95%, and 90% confidence levels, respectively.Higher confidence levels increase spinning-reserve costs, creating a reliability–economy trade-off.
  • The proposed scheduling approach reduces operating costs and achieves faster calculation than traditional scheduling without forecasting.The authors attribute the economic improvement to integrating PER-AutoRL forecasting into scheduling.
Loading 2108.06764v1…