PaperlayerView exact revision on arXiv

Source-linked AI summary

Ancillary Service to the Grid Using Intelligent Deferrable Loads

Sean Meyn, Prabir Barooah, Ana Bušić, Yue Chen, Jordan Ehren

arXiv:1402.4600v1math.OCeess.SY

TL;DR

Renewable volatility creates a demand-supply balancing challenge that flexible loads may help address. The paper combines decentralized randomized MDP control with a mean-field aggregate model and LTI approximation, applying the approach to residential pool pumps. The approximation is reported as remarkably accurate, while consumer energy costs and risks remain incompletely understood.

  • Problem

    Renewable volatility complicates demand-supply balancing, while the potential of flexible loads for ancillary service remains largely untapped.

  • Method

    The paper develops decentralized randomized MDP control for individual loads, a mean-field aggregate model, and an LTI approximation controlled through a broadcast scalar signal.

  • Results

    The LTI approximation is reported as remarkably accurate for the mean-field model and is applied to pool pumps for ancillary service.

  • Takeaways & Limitations

    The approach provides a convenient basis for grid-level classical control while concentrating intelligence at individual loads.

  • Takeaways & Limitations

    The paper does not fully understand the potential consumer energy cost or associated risk.

Abstract

from arXiv · show

Renewable energy sources such as wind and solar power have a high degree of unpredictability and time-variation, which makes balancing demand and supply challenging. One possible way to address this challenge is to harness the inherent flexibility in demand of many types of loads. Introduced in this paper is a technique for decentralized control for automated demand response that can be used by grid operators as ancillary service for maintaining demand-supply balance. A Markovian Decision Process (MDP) model is introduced for an individual load. A randomized control architecture is proposed, motivated by the need for decentralized decision making, and the need to avoid synchronization that can lead to large and detrimental spikes in demand. An aggregate model for a large number of loads is then developed by examining the mean field limit. A key innovation is an LTI-system approximation of the aggregate nonlinear model, with a scalar signal as the input and a measure of the aggregate demand as the output. This makes the approximation particularly convenient for control design at the grid level. The second half of the paper contains a detailed application of these results to a network of residential pools. Simulations are provided to illustrate the accuracy of the approximations and effectiveness of the proposed control approach.

I. INTRODUCTION

Rising renewable volatility increases the need for balancing resources, while flexible deferrable loads offer automated, decentralized ancillary service with limited consumer impact. The paper develops randomized MDP-based control and an LTI aggregate approximation, then applies it to residential pool pumps.

  • Renewable generation increases demand-supply balancing challenges, motivating additional resources for large power fluctuations.
  • Flexible loads can provide ancillary service without central control or significant impact on consumers or industry.
  • Deferrable loads are attractive ancillary-service resources because the service targets zero average energy consumption.
  • The proposed architecture uses randomized control to support decentralized decisions and avoid synchronization that can create detrimental demand spikes.
  • An individual-load MDP and mean-field aggregate model lead to an LTI approximation whose scalar input is suitable for grid-level control.
  • Residential pool pumps are used as a medium-frequency ancillary-service application, with simulations assessing approximation accuracy and control effectiveness.

II. OPTIMAL CONTROL FOR A LOAD AND FOR THE GRID

The paper develops a decentralized control architecture in which a balancing authority broadcasts a scalar command to randomized loads, enabling aggregate demand tracking while avoiding synchronization. It constructs load-level dynamics and an LTI input-output approximation for grid-level control.

  • Control architecture: A balancing authority computes a common scalar command ζ and transmits it to individual loads, whose binary decisions depend on local state and ζ.The architecture distributes control decisions while retaining centralized measurement and command generation.
  • Control architecture: Randomization avoids synchronization among loads and facilitates analysis of the aggregate system.The approach is motivated by decentralized decision making and the risk of synchronized demand spikes.
  • Grid-level objective: The balancing authority tracks aggregate consumption near y0 + r by regulating the deviation ey = y − y0 through ζ.The tracking problem is addressed using classical control techniques when the ζ-to-ey dynamics admit an LTI approximation.
  • Load-level design: The controlled transition-matrix construction balances grid-level aggregate dynamics against each load’s quality-of-service requirements.For pools, these requirements include clean water and a constant electricity bill over each period.
  • Approximation and application: The resulting aggregate nonlinear dynamics yield an LTI approximation from ζ to ey, supporting classical grid-level control design.For residential pools, the approximation is minimum phase and a simple PI controller is effectively used.

B. Load model and design

The load model represents pool-pump operation as a controlled finite-state Markov chain and derives its policy by optimizing welfare relative to nominal behavior. The infinite-horizon solution is characterized through an eigenvector problem.

  • Load model: The nominal transition matrix P0 models control-free behavior, while the controlled chain introduces randomness around typical pump schedules.Most consumers would otherwise run pumps for a fixed number of hours each day.
  • Load model: Each pool pump is modeled on a finite state space encoding whether it is on or off and the duration of its current operating mode.States are X = {(m, i): m ∈ {⊕, ⊖}, i ∈ {1, ..., T}}.
  • Welfare optimization: The utility function indicates whether the pump is operating, and welfare balances average utility against the cost of deviation.The scalar ζ serves as the weighting parameter in this welfare objective.
  • Welfare optimization: The finite-horizon optimizer is a twisted probability distribution whose optimal value equals ΛT(ζ).The resulting distribution defines a Markov chain, though it need not be time-homogeneous over a finite horizon.
  • Infinite-horizon design: For irreducible P0, the infinite-horizon optimizer is a time-homogeneous Markov chain computed from the unique maximal positive eigenvalue-eigenvector pair.The resulting Markov model achieves the optimal average welfare.

C. Approximations

The paper develops Taylor approximations for the controlled Markov model near ζ = 0 using nominal-model quantities, including Poisson-equation solutions and asymptotic variance. These formulas support analysis but are not directly computational.

  • Approximation setup: The approximation analysis is needed when the scalar command ζ changes over time.The nominal model P0 supplies the expectations and invariant distribution used in the expansion.
  • Approximation setup: The first-order coefficient is the nominal steady-state mean, while the second-order coefficient uses the nominal model’s asymptotic variance.The variance is the quantity appearing in the central limit theorem for the nominal model.
  • Approximation construction: Poisson’s equation provides the first-order term and an expression for the asymptotic variance in the finite-state model.The relative value function and related quantities are expanded through these representations.
  • Approximation results: ζ is convex and admits a Taylor expansion, while the invariant-measure mean and relative value function receive corresponding first- and second-order approximations.These results are collected in Proposition 2.3 for irreducible P0.
  • Computational scope: The analytical representations are useful for analysis but not for computation.The paper notes that computational methods for H and S are contained in the Appendix.

D. Aggregate load model

The aggregate model takes a mean-field limit of many independently controlled loads and then linearizes the resulting nonlinear distribution dynamics around nominal equilibrium. This produces a finite-dimensional linear state-space model for aggregate demand control.

  • Aggregate formulation: The aggregate control problem regulates average utility using measurements of aggregate consumption and a desired regulation signal.The observed output yt is the fraction of loads that are on.
  • Mean-field model: With N tending to infinity, the empirical load distribution is represented by a controlled distribution process µt driven by the command sequence ζ.The mean-field limit is justified by a stated theorem and law-of-large-numbers arguments.
  • Linear approximation: Taylor approximations reduce the nonlinear aggregate system to a linear state-space model with d-dimensional state Φ and output γ.The resulting model is constructed as the second step after the mean-field formulation.
  • Linear approximation: The nonlinear distribution dynamics evolve as µt+1 = µt P̌ζt, with equilibrium µt = π0 and yt = y0 when ζ is identically zero.Proposition 2.4 linearizes these dynamics around the nominal equilibrium.
  • Linear approximation: The linearized model uses output weights from the utility function, an input matrix derived from the controlled transition dynamics, and an initial deviation from π0.Its approximation leads to the paper’s input-output formula from ζ to aggregate demand deviation.
  • Mean-field justification: The finite-load empirical distributions equal the mean-field model plus a disturbance that vanishes as N tends to infinity.The convergence argument uses bounded-increment martingale laws and a law-of-large-numbers result.

III. CONTROLLING A LARGE NUMBER OF POOLS

The paper applies its control framework to a large population of residential pools, specifying pool switching behavior and a welfare-based control parameter. Numerical examples examine symmetric and asymmetric cleaning-period models.

  • Pool model: A large population of residential pools is modeled using a nominal transition matrix P0 whose probabilities describe pump switching on and off.The transition probabilities depend on how many hours the pump has been off or on.
  • Pool model: The pool utility function is the indicator that the pump is operating.
  • Control parameter: Positive ζ provides an incentive to turn pumps on.
  • Numerical setup: For T = 30 minutes over a 24-hour day, the symmetric model uses the corresponding sampling parameters for numerical studies.
  • Numerical setup: As γ →∞, the control functions converge to step functions corresponding to a deterministic cleaning period of α × 24 hours.The average cleaning period is found numerically to be somewhat smaller in the reported model.

A. Approximations

The controlled pool model admits linear and quadratic approximations for steady-state operation probability and related quantities. The probability approximation is especially accurate near the central range of ζ and saturates for larger positive values.

  • Steady-state probability: The steady-state probability that a pool pump is operating is approximated using the paper’s linear and quadratic expansions.These approximations are compared with the true controlled probability.
  • Steady-state probability: The affine approximation is very tight for |ζ| ≤3 in the symmetric model with η0 = 1/2.
  • Steady-state probability: For larger ζ, the true steady-state probability saturates at approximately 0.9 as ζ →+∞.
  • Controlled transition probabilities: The controlled transition matrix P̌ retains the form of P0 while using transformed probability vectors P̌p⊕ and P̌p⊖.Plots show P̌p⊕ for ζ = 0, ±2, ±4, with P̌p⊖ obtained by symmetry.
  • Approximation accuracy: The approximation is nearly perfect over ζ ∈[−4, 4].The cited passage refers to the approximation shown for the relevant modeled quantity.

B. Aggregate load model for pool population

The aggregate pool model uses a linear state-space representation for control synthesis and relates desired aggregate consumption to a target steady-state pool-on probability and control value. Its transfer function is BIBO stable and minimum phase.

  • Aggregate model: The paper uses the linear model (26) as the basis for control synthesis by the balancing authority.
  • Equilibrium analysis: The steady-state pool-on probability is defined as y∞= limt→∞yt for the model with transition law P̌ζ∗.
  • Aggregate model: Desired aggregate consumption G∗ determines the target steady-state probability y∞= G∗/(Ngp) for a population of N pools.Here gp is the consumption of one operating pool pump.
  • Aggregate model: The target control value ζ∗ is defined relative to the control-free consumption G0 = gpNη0 through eG = G∗−G0.
  • Dynamic properties: The transfer function from ζ to γ is BIBO stable and minimum phase.The paper presents its Bode and pole-zero plots for the linear model.

C. Super-sampling

The paper refines randomized control with super-sampling to reduce aggregate response delay without requiring pools to share a clock. Pools are divided into timing classes, producing a low-pass-filtered aggregate response.

  • Motivation: The refinement addresses the unacceptable delay that arises when the control model updates hourly despite nearly instantaneous changes in power consumption.
  • Super-sampling design: Super-sampling uses a grid-level sampling interval T/m, where m > 1 is the super-sampling parameter.For T = 30 minutes and m = 6, the interval is five minutes.
  • Super-sampling design: Pools have no common clock and are assigned to class i according to when they check the regulation signal within each interval T.Class i checks at nT + (i −1)T/m for n ≥0 and 1 ≤i ≤m.
  • Aggregate response: The aggregate number of operating pools is formed by summing the operating fractions across timing classes.
  • Aggregate response: Each timing bin contributes a fraction 1/m of total ancillary service, and the resulting transfer function includes a low-pass filter L.The filter has m −1 unit-circle zeros, excluding z = 1, after a pole-zero cancellation at z = 1.
  • Outcome: The real-time delay is reduced from T to T/m using super-sampling.

D. Simulation results

Simulations of one million residential pools show that the proposed control can track regulation signals, while available capacity depends strongly on the cleaning schedule and pool operating fraction.

  • Experiments: One million pools could provide far more regulation than the ±200 MW required at BPA during the tested week.The experiments used stochastic simulation and assumed perfect measurements of aggregate pool power consumption.
  • Approximation properties: The linearization is minimum phase in both scenarios, while supersampling introduces zeros on the unit circle in the resulting transfer function.All zeros of H0(z) lie strictly within the unit disk before supersampling.
  • Operating balance: Average pool operation is approximately 1/2 in Scenario 1 and 1/3 in Scenario 2, giving Scenario 2 more potential to increase than decrease consumption.The regulation-signal class is therefore asymmetric under the shorter schedule.
  • Capacity: Potential capacity is {+500MW, −500MW} in Scenario 1 and {+695MW, −305MW} in Scenario 2.Capacity is defined as the upper and lower power-deviation limits subject to no degradation in tracking performance.
  • Conclusions and open issues: The authors report remarkable accuracy of the LTI approximation and identify unresolved issues concerning consumer costs, rare cleaning events, measurements, and customer contracts.The paper notes that frequency measurements may suffice, but consumer preferences and ancillary-service value require better understanding for contracts.

APPENDIX

The appendix develops finite-state Markov-chain arguments for invariant measures, Poisson equations, eigenvalue identities, and convergence bounds supporting the paper's analysis.

  • Spectral structure: Perron–Frobenius theory provides a unique maximal eigenvalue and positive eigenvector for the finite-state Markov chain.The appendix uses this structure to establish the relevant spectral representation.
  • Asymptotic analysis: The identity Λ = log(λ) = η*ζ links the maximal eigenvalue to the asymptotic quantity used in the analysis.The appendix states that this identity follows from the cited multiplicative ergodic theory result.
  • Convergence bounds: The appendix derives bounds on finite-horizon costs and their convergence to the infinite-horizon limit.The proof combines probability-measure identities with Proposition 2.1 and related bounds.
  • Poisson equations: Poisson equations are used to characterize and compute the functions needed in the second-order analysis.The finite state space permits solution through elementary matrix algebra.
  • Invariant measure: The invariant probability measure is obtained by normalizing an unnormalized invariant measure expressed through a matrix inverse.The construction uses the minorization condition and the resolvent [I − P]^-1.

2) Second-order approximation:

The second-order approximation differentiates the parameterized eigenvector equations and represents the resulting terms through Poisson equations and a nonlinear variance operator.

  • Approximation setup: The Taylor-series approximation is developed with respect to the forcing signal ζ, whose dependency is suppressed in the proof notation.The appendix treats ζ as a forcing function analogous to the earlier equation.
  • Second-order term: The second-order term S is shown to solve a Poisson equation that can be computed using elementary matrix algebra.The representation for h''ζ is described as appearing to be new.
  • Derivation tools: The nonlinear generator and the operator V(g) = Pg^2 − [Pg]^2 support derivations of the eigenvector and variance-related equations.The generator is defined for arbitrary functions, and the eigenvector equation is written in a Poisson-equation-like form.
  • Derivative calculations: The appendix obtains first- and second-derivative relations by differentiating the eigenvector equation and applying product-rule calculations.The resulting expressions are organized in equations (43), (47), and (48).
Loading 1402.4600v1…