Source-linked AI summary
Learning to Price Electricity for Optimal Demand Response
Jing Shang, Mohammad Mehrabi, Xinyang Zhou, Mahmoud Saleh, Andrey Bernstein, Stefan Wager
TL;DR
Renewable production can be misaligned with demand, while constrained-output learning methods present challenges for feasible pricing. The paper learns feasible contextual prices with a cluster-based neural architecture and reports larger demand-management benefits in solar-rich, variable settings.
Problem
Renewable supply can be misaligned with demand, and existing constrained-output learning methods are not directly deployable for the pricing setting.
Source: §1 · pp. 2–3Method
A cluster-based neural network learns context-dependent convex combinations of feasible prototype prices from operational feedback.
Source: §1 · p. 3, §2.1 · p. 9Results
Contextual pricing delivers substantial demand-management gains over flat-rate and context-agnostic dynamic pricing, with larger benefits in solar-rich regions with high variability.
Source: §1 · p. 4, §7 · p. 21Takeaways & Limitations
Learning a constrained mapping from contextual signals to feasible prices can generate near-optimal prices with minimal computational overhead once trained.
Source: §7 · p. 21Performance may degrade when contextual features are mis-specified or poorly predicted, and finite-population effects may cause behavior synchronization and deviations from optimality.
Source: §7 · p. 22
Abstract
from arXiv · showhide
There is considerable interest in using time-varying electricity prices to shape consumer demand response, and better align energy demand with renewable production. However, optimal prices generally vary over time in response to complex signals such as weather forecasts, sunrise/sunset times, and day-of-week patterns; and existing methods are not able to make efficient use of such rich contextual information. Here, we propose a neural-network-based algorithm for contextual energy pricing, modeling pricing as a Stackelberg game and leveraging a mean-field solution representation from Mehrabi et al.~(2024). The approach learns constrained mappings from contextual features to feasible price signals. We validate our approach by simulating the energy grid in several US cities, and show that incorporating contextual information can considerably increase the value of the demand response programs.
1 Introduction
Context-aware pricing is motivated by renewable supply that varies independently of demand, while flexible household loads offer opportunities to shift consumption. The paper introduces a constrained neural pricing method and reports demand-management gains, especially in solar-rich, variable settings.
1 Introduction
- Renewable production can misalign with demand, while flexible residential loads—including HVAC, water heating, appliances, and EV charging—can shift consumption across hours without sacrificing energy goals.The residential sector accounts for approximately one-third of U.S. retail electricity consumption.Source: §1 · p. 2
- Fixed price schedules can manage demand poorly when conditions vary: cloudy-day prices applied on a sunny day produce a pronounced duck curve and less stable net demand.The paper contrasts optimal sunny- and cloudy-day schedules in Figure 1.Source: §1 · p. 2, Figure 1 · p. 3
- The paper models daily contextual pricing as an online process: the planner observes day-specific features, deploys prices, and observes consumer behavior under the indifference-set approach of Mehrabi et al. (2024).Earlier work found optimal demand-response benefits in its model but addressed a time-homogeneous setting.Source: §1 · pp. 2–3
- A cluster-based network forms context-dependent convex combinations of feasible prototype prices, guaranteeing feasible outputs while permitting first-order updates.The design responds to limitations of projection, penalty, and optimization-layer approaches to constrained-output learning.Source: §1 · p. 3, §1.1 · p. 5
- Contextual pricing delivers substantial demand-management gains over flat-rate and context-agnostic dynamic pricing, with larger benefits in solar-rich regions with high variability.The evaluation spans multiple cities and heterogeneous grid and consumer conditions.Source: §1 · p. 4
- The proposed architecture has a universal approximation guarantee for contextual mappings with compact output constraints, and its first-order updates converge to a stationary point of a context-free clustering problem.The convergence analysis concerns standard back-propagation for network weights and projected gradient descent for prototypes.Source: §1 · p. 4
1.1 Related work
- Prior contextual pricing work includes predictive wholesale-market models and mechanisms conditioned on a small number of states; this paper targets a smooth context-to-incentive mapping for demand management.The related-work discussion distinguishes predictive price forecasting from incentive design.Source: §1.1 · p. 5
- Unlike approaches requiring counterfactual demand, detailed household models, or repeated communication, the proposed method learns from realized demand at deployed prices.The paper says it does not require internal household energy-use knowledge or imputed counterfactual demand.Source: §1.1 · pp. 5–6
2 Pricing for optimal demand response
This section develops a feasible, context-dependent pricing approach for demand response, modeling shifts among service-equivalent consumption schedules and learning prices from operational feedback. It updates a contextual prototype-based rule under price constraints and evaluates required non-renewable generation through grid simulation.
2 Pricing for optimal demand response
- Consumers choose among service-equivalent load profiles that meet energy goals, so prices shift consumption timing rather than suppressing underlying demand.The framework models the grid operator as announcing prices and households as minimizing energy costs over their indifference sets.Source: §2 · p. 6
- Convex, positively homogeneous grid-risk functions include norm-based objectives such as total net demand under ℓ1 and peak demand under ℓ∞.Such risk functions can be represented through a dual feasible price set.Source: §2 · p. 7
- Mehrabi et al. (2024) characterize the mean-field optimum as a concave maximization over prices, reformulating the challenging bilevel pricing problem.The characterization uses average consumer cost and the feasible price set induced by the grid-risk function.Source: §2 · p. 7
2.1 Contextual pricing
- Context changes renewable supply and household cost distributions, while prototype prices with contextual weights map each context to a feasible price.The demonstrated covariates include temperature and solar-radiation forecasts, as well as day-of-week and holiday effects.Source: §2.1 · pp. 8, 10
- Observed aggregate demand response supplies first-order updates for contextual weights and price prototypes without requiring the operator to observe household preferences.The operator uses realized net demand from consumption and renewable-supply observations to update the rule sequentially.Source: §2.1 · pp. 8–9
- Forecast errors can make planned and realized consumption differ, adding load variability and potentially affecting the learned pricing policy’s performance.For example, higher-than-forecast outdoor temperatures can increase cooling consumption beyond the planned level.Source: §2.1 · p. 11, Figure 3 · p. 12
- The contextual pricing rule updates its weights using Theorem 2.1’s first-order update and projects prototype updates onto the feasible set P.The rule is p_t = Z_tπ_θt(X_t).Source: §2.1 · p. 12
- Feasibility follows by construction: prototypes lie in P, network outputs lie in the probability simplex, and convexity keeps normalized prices in P.The normalized price is e p_t = Z_tπ_θt(X_t).Source: §2.1 · p. 12
- Required non-renewable generation Δ_t is obtained from an OpenDSS grid simulation and incurs generation cost ρ(Δ_t).Detailed power-system settings are referred to Section 3.2.Source: §2.1 · p. 12
3 Simulator design
The simulator models 486 HEMS-equipped households on a representative suburban feeder, with device-specific feasible consumption profiles and a grid cost that captures both aggregate consumption and temporal variation.
3 Simulator design
- The simulator uses a SMART-DS distribution network with 486 HEMS-equipped households arranged as a typical suburban neighborhood, capturing neighborhood and substation behavior.Device configurations are described in Zhou et al. (2026).Source: §3 · p. 12
3.1 Household device modeling
- Household flexibility combines fixed lighting, forecast-constrained HVAC scheduling, and BESS charging and discharging constrained by state of charge.The aggregate household indifference set combines the feasible sets of its devices.Source: §3.1 · p. 13
3.2 Grid generation cost
- Grid generation cost captures aggregate consumption and temporal variation via first differences, while each simulation step projects prices onto the dual-norm price set.The projection uses a bisection method when the unconstrained price lies outside the set.Source: §3.2 · pp. 13–14
4 Algorithm evaluation
The evaluation tests contextual pricing in alternating weather and benchmarks contextual-learning methods. The cluster-based soft classifier learns regime-specific prices and leads the benchmark methods in peak demand shaving.
4.1 Learning heterogeneity
- With K = 2, the cluster-based classifier learns distinct prices for the two weather regimes, while feedback-only pricing converges to approximately their average.The alternating regimes pair high temperatures and low solar irradiance with low temperatures and high solar irradiance.Source: §4.1 · pp. 14–15
- Contextual pricing consistently achieves greater peak demand shaving than feedback-only pricing under alternating weather conditions.Using forecast information, the contextual method adapts to day types, while feedback-only pricing takes several days to adjust.Source: §4.1 · p. 15, Figure 5 · p. 16
4.2 Benchmark algorithms
- The benchmark compares four contextual-learning algorithms on the same l2-norm grid-risk objective using 90 days of Denver weather data.The evaluation also uses historical weather and forecast data for its simulations and case study.Source: §4 · p. 14, §4.2 · p. 15
- The cluster-based soft classifier achieves the highest average peak demand reduction with the lowest variability, and is the only method consistently achieving positive peak demand shaving.Projection-layer and interior-point methods are volatile and sometimes produce negative peak demand shaving; proximal gradient performs second best.Source: §4.2 · p. 16
5 Case study: value of demand response across cities
Across San Francisco, Atlanta, and Phoenix, contextual pricing produces positive peak demand shaving on most days relative to static pricing. The magnitude varies with climate-related demand flexibility, and learned prices track city-specific temperature patterns.
5 Case study: value of demand response across cities
- The case study examines three cities with distinct summer climates, using temperature and solar-irradiance profiles to characterize their differing system conditions.The cities are San Francisco, Atlanta, and Phoenix; Figure 8 shows monthly mean daily profiles and day-to-day variability.Source: §5 · pp. 16–17, Figure 8 · p. 18
- Peak demand shaving is positive on the vast majority of days across the three cities, with only a small number of exceptions, and improves load shaping relative to static pricing.The case study assumes households have home energy management systems coordinating lighting, HVAC, and battery storage.Source: §5 · pp. 17–18
- Average peak demand shaving is highest in Atlanta and lowest in San Francisco, with results depending on climate-related demand flexibility.Mild San Francisco summers limit air-conditioning load, while Phoenix heat constrains HVAC flexibility through comfort requirements; Atlanta’s moderate, variable cooling demand offers more flexibility.Source: §5 · p. 18
- Across the three cities, average hourly price profiles mirror average hourly temperature patterns, with sharper price increases during hotter periods and flatter profiles in milder climates.Figure 10 summarizes the learned price profiles over 30 days in July.Source: §5 · p. 19, Figure 10 · p. 19
6 Theoretical properties
The proposed architecture has universal approximation capacity for constrained context-to-output mappings and learning dynamics that converge to stationary points of an underlying clustering problem. These results hold under the stated compactness and regularity conditions.
6.1 Universal approximation
- The approximation argument represents outputs as convex combinations of constrained prototypes, with softmax networks approximating the required cluster assignments.The proof uses simple-function approximation to refine regions of the input space.Source: §6.1 · pp. 19–20
- With sufficiently many clusters and hidden units, the architecture can approximate any measurable mapping into a compact constraint set arbitrarily well.For continuous mappings on compact X, the convergence can be uniform.Source: §6.1 · pp. 19–20
6.2 Convergence analysis
- Under mild regularity conditions, the contextual network updates converge to a first-order stationary point of the underlying context-free clustering problem.If the stationary points are isolated, the iterates converge to a single such point.Source: §6.2 · p. 21
7 Discussion
The discussion presents contextual pricing as a way to learn feasible, context-dependent prices without repeatedly solving a bilevel problem for each observed state. It reports stronger performance than flat-rate and context-agnostic prices in settings with high renewable penetration or substantial day-to-day weather variation, while noting limitations and future extensions.
7 Discussion
- The framework improves over flat-rate and context-agnostic prices particularly with high renewable penetration or substantial day-to-day weather fluctuations.It learns a constrained mapping from contextual signals to feasible prices using a mean-field characterization of Stackelberg equilibrium.Source: §7 · p. 21
- Learning a contextual price mapping avoids repeatedly solving a bilevel optimization problem for each observed state and can generate near-optimal prices with minimal computational overhead once trained.The discussion identifies this distinction as relevant to real-time or large-scale implementation.Source: §7 · p. 21
- The framework depends on accurate, representative contextual features, and finite-population effects may cause synchronization and deviations from optimality.The mean-field guarantees are asymptotic.Source: §7 · p. 22
- Future work could model uncertainty in contextual features and strategic consumers who anticipate pricing policies and optimize intertemporal utility.Suggested approaches include probabilistic forecasts or Bayesian neural networks for input uncertainty.Source: §7 · p. 22
A Approaches for constrained-output learning
- The appendix reviews projection-based, barrier/interior-point, and proximal primal-dual methods, and proposes a cluster-based soft classifier that constrains cluster outputs.The classifier maps covariates to cluster weights while learning constrained cluster-level outputs.Source: A Approaches for constrained-output… · p. 25
A.1 Soft classifier architecture
The soft-classifier architecture learns context-dependent weights over feasible cluster outputs, so their convex combination remains feasible. The appendix also describes its update procedure and contrasts it with projection-layer, interior-point, and proximal-gradient alternatives.
A.1 Soft classifier architecture
- The network assigns context-dependent probabilities to clusters, and constraints on cluster outputs ensure that their convex combination is feasible.Each cluster represents an archetypal system condition.Source: §A.1 · pp. 25–26, Figure 11 · p. 26
A.2 Algorithmic update
- Network parameters are updated by unconstrained gradient descent, while constrained cluster outputs are updated by projected gradient descent.The two parameter groups are updated sequentially.Source: §A.2 · pp. 26–27
A.3 Projection layer
- A projection layer enforces feasibility by projecting network outputs directly, whereas projected gradient descent projects training variables during optimization.The projection-layer approach can make gradients challenging when the projection operator is nondifferentiable and may increase overfitting risk.Source: §A.3 · p. 27, Figure 12 · p. 28
A.4 Interior point
- The interior-point method adds a log-barrier penalty to the loss, but can be numerically unstable near the constraint boundary and requires challenging tuning of its penalty parameter.It also fails when the barrier function is nondifferentiable.Source: §A.4 · p. 28
A.5 Proximal gradient
- Proximal gradient uses weak Lagrange duality to accommodate a broader class of constraints, including non-differentiable ones.The method updates the dual variable with proximal gradient descent, projecting it onto the box [−λ, λ]d.Source: §A.5 · pp. 28–29
B Extended case study analysis
Across the three cities, contextual pricing reduces peak demand consistently, including on high-demand days. The reported net-peak comparisons show no evidence that the program exacerbates peaks during critical periods.
B Extended case study analysis
- Net peak demand differs substantially across San Francisco, Atlanta, and Phoenix, with city patterns aligning with their temperature and solar-irradiance patterns.Figure 13 compares net peak demand under flat-rate and context-aware pricing.Source: B Extended case study analysis · p. 30
- Peak demand reductions are consistent across cities and days, remain pronounced during high-demand periods, and show no evidence of exacerbating peaks at critical times.Most observations at high demand levels lie below the 45-degree line, although such observations are relatively sparse.Source: B Extended case study analysis · pp. 29–30
C Proof of Theorem 2.1
The proof derives a contextual price update from the mean-field optimal-price characterization. It uses average consumption as first-order information, with a supergradient interpretation when household responses are non-unique.
C Proof of Theorem 2.1
- The optimal price is characterized using average consumer cost and a subgradient based on consumers’ consumption profiles.The characterization is borrowed from Mehrabi et al. (2024).Source: C Proof of Theorem 2.1 · p. 30
- Figure 13 compares daily net peak demand under flat-rate and context-aware pricing across cities in 2022; points below the diagonal indicate lower demand with the proposed framework.Source: Figure 13 · p. 31
- The mean-field average consumption response is a supergradient of the concave consumer cost function in price.For unique household responses, the selected-response expression is the usual gradient; otherwise, it is a valid supergradient.Source: C Proof of Theorem 2.1 · p. 31
- Applying the chain rule to the finite-sample objective yields price-dependent first-order updates through the induced contextual price map.The proof uses the transposed Jacobian of that map with respect to the network parameters.Source: C Proof of Theorem 2.1 · p. 31
D Proof of Theorem 6.3
The proof constructs feasible approximations to measurable contextual mappings by first approximating them with simple functions and then ensuring their values lie in the constraint set. For continuous mappings on compact domains, the construction can be uniform.
D Proof of Theorem 6.3
- Any measurable mapping into the compact constraint set can be approximated by functions of the proposed form.The construction begins with coordinate-wise simple-function approximations and forms vector-valued simple functions.Source: D Proof of Theorem 6.3 · p. 32
- The approximating functions retain feasibility by choosing their values in P, so each constructed function maps into P.The proof selects feasible values for each partition element.Source: D Proof of Theorem 6.3 · p. 32
- For a continuous mapping on a compact domain, the approximations can be constructed uniformly using uniform continuity.Source: D Proof of Theorem 6.3 · p. 32
E Convergence analysis of prototype learning
This section analyzes convergence of prototype-based neural networks for hard-constraint output learning. Under stated assumptions, their iterates approach stationary points of the context-free clustering problem, and isolated stationary points yield convergence to a single point.
E Convergence analysis of prototype learning
- The clustering loss is separately convex in mixture weights and prototypes, but not jointly convex because the two are multiplicatively coupled.Source: E Convergence analysis of prototype… · p. 33
- An entropic term makes the weights uniquely optimal for each fixed prototype set, including in overparameterized settings where the unregularized problem may not be identifiable.The term keeps weights in the simplex’s relative interior and makes the objective strongly convex in the weights for fixed prototypes.Source: E Convergence analysis of prototype… · p. 33
- The architecture represents each response using context-dependent mixture weights over K prototypes constrained to P.The weights lie in the probability simplex, and the prototypes are selected to explain the response vectors in the context-free clustering objective.Source: E Convergence analysis of prototype… · p. 33
- Under the theorem’s assumptions, prototype-based iterates approach first-order stationary points of the context-free clustering problem; the feature-conditioning assumption requires N ≥ n.The theorem assumes diminishing step sizes, and the conditioning condition forces the overparameterized regime.Source: E Convergence analysis of prototype… · pp. 33–34
- If the stationary points are isolated, the iterates converge to a single stationary point; isolation is a property of the data distribution, not the architecture.Source: E Convergence analysis of prototype… · pp. 34–35
E.1 Proof of Theorem E.1
The proof establishes convergence by showing smoothness and vanishing gradients for the composed loss, then demonstrating that every limiting point is stationary. Compactness converts this into convergence of the iterates’ distance to the stationary set.
E.1 Proof of Theorem E.1
- The composed loss is twice continuously differentiable and has an L-Lipschitz gradient on the compact parameter set.Source: §E.1 · p. 35
- The relevant gradients vanish under the stated assumptions and diminishing step sizes.Source: §E.1 · pp. 35–36
- The proof applies the chain rule to relate the mixture-weight gradient to the network-parameter updates, while the prototype update uses its projected optimality condition.Source: §E.1 · pp. 35–36
- Every convergent subsequence has a limit in Cn, and compactness then implies that the distance of the iterates to Cn tends to zero.Source: §E.1 · p. 36
E.2 Proof of Lemma E.3
The proofs establish smoothness and a compact-set Lipschitz bound for the composed loss, then use smoothness and projected updates to show the gradient and normalized Z-step residual vanish.
E.2 Proof of Lemma E.3
- The smoothness proof composes σ(XU)θ with soft, then log S, while the remaining F term is polynomial in S and Z.Source: §E.2 · pp. 36–37
- The composed loss is twice continuously differentiable at finite points, and its Hessian has a finite maximum on compact K, making its gradient L-Lipschitz there.Source: §E.2 · p. 37
E.3 Proof of Lemma E.4
- Assumption 3 keeps the trajectory in a fixed compact convex set, and after some t0 the step size satisfies η_t≤1/L.Source: §E.2 · p. 37, §E.3 · p. 38
- For η_t≤1/L, the (U, θ)-only update decreases the loss by η_t times the squared gradient norm before Z is updated.Source: §E.3 · p. 37
- The proof establishes that the gradient and the normalized Z-update step converge to zero as iterations grow.The quantities are ∇_{U,θ}ℓ_n and ||Z_{t+1}−Z_t||_F/η_t.Source: §E.3 · pp. 38–39