Source-linked AI summary
Stochastic maximum principle for optimal control of SPDEs
Marco Fuhrman, Ying Hu, Gianmario Tessitore
TL;DR
General stochastic maximum-principle results for nonconvex controls and control-dependent diffusion have limited applicability to concrete SPDEs. The paper gives a semi-abstract approach with a second-variation adjoint, and shows its conclusions apply to a class of parabolic SPDEs, including a stochastic heat-equation setting.
Problem
For infinite-dimensional stochastic control, general results for nonconvex controls and control-dependent diffusion use assumptions not met by most concrete SPDEs.
Source: §1 · p. 1Method
The paper formulates the controlled equation semi-abstractly and characterizes the second-variation adjoint as an operator-valued process through a stochastic bilinear form.
Source: §1 · pp. 2–3Results
The theorem's conclusions apply to parabolic SPDEs with a Laplacian operator and appropriate boundary conditions, under the stated hypotheses.
Source: §3.2 · p. 6Takeaways & Limitations
The results cover, for instance, a controlled stochastic heat equation with finite-dimensional noise.
Source: §1 · p. 2, §3.2 · p. 6The stated maximum principle assumes an analytic semigroup on L^4 whose generator domain is compactly embedded in L^4.
Source: §3.1 · p. 5
Abstract
from arXiv · showhide
We prove a version of the maximum principle, in the sense of Pontryagin, for the optimal control of a stochastic partial differential equation driven by a finite dimensional Wiener process. The equation is formulated in a semi-abstract form that allows direct applications to a large class of controlled stochastic parabolic equations. We allow for a diffusion coefficient dependent on the control parameter, and the space of control actions is general, so that in particular we need to introduce two adjoint processes. The second adjoint process takes values in a suitable space of operators on $L^4$.
1 Introduction
The infinite-dimensional stochastic maximum principle remains difficult to apply to concrete SPDEs in the general case of nonconvex controls and control-dependent diffusion. The paper addresses this gap with a semi-abstract formulation and a second-adjoint construction intended to cover controlled parabolic equations.
1 Introduction
- The maximum-principle problem for infinite-dimensional stochastic control was addressed in early work, but important issues of generality and SPDE applicability remain.The finite-dimensional analogue was treated in generality by Peng.Source: §1 · p. 1
- Existing results for nonconvex controls and control-dependent diffusion rely on assumptions that exclude most concrete SPDEs.Spike variation requires a second variation when the control enters the diffusion, whose adjoint is difficult to characterize in infinite dimensions.Source: §1 · pp. 1–2
- The results cover controlled stochastic heat equations with finite-dimensional noise, including a Laplacian example with appropriate boundary conditions.The state equation is formulated semi-abstractly to address concrete nonlinearities while using the leading elliptic operator's regularizing properties.Source: §1 · p. 2
- Earlier work treated diffusion-independent cases and linear state equations with linear costs; addressed the general case but imposed regularity conditions that hinder SPDE applications.The paper also contrasts its scope with [10], [3], and [4], whose coefficient regularity assumptions are described as too restrictive for direct application to controlled SPDEs.Source: §1 · p. 2
- The paper develops a second-variation adjoint as a progressive process of bounded operators from L^4 to L^4/3, defined through a stochastic bilinear form.Its regularity analysis and estimates for the first and second variations are central technical issues.Source: §1 · p. 3
2 Notations and preliminaries
The preliminaries formulate the controlled stochastic equation on L^p spaces over a finite-measure domain and specify the control and coefficient assumptions. They establish mild-solution existence and higher-moment regularity using semigroup bounds and contraction arguments.
2 Notations and preliminaries
- The state equation has a semigroup-driven drift, controlled drift and diffusion, and an initial condition; controls are progressive processes valued in a separable metric space.The state and cost are used to define an optimal pair by minimizing the cost over all control processes.Source: §2 · p. 3
- The coefficient functions φ (b, l, and σj) and h are measurable, and their dependence on the state variable r has continuous first and second derivatives.Source: §2 · p. 4
- For every control process, the state equation has a unique mild solution in L^2 with continuous trajectories, and its trajectories satisfy higher L^p moment bounds.The higher-summability result applies for p in [2, p̄].Source: §2 · p. 4
- The existence proof uses a contraction on progressive L^p-valued processes for sufficiently short time intervals, then extends the result across the full horizon by subdivision.The contraction estimate uses L^p-boundedness of the semigroup and the Lipschitz property of the diffusion coefficient.Source: §2 · pp. 4–5
3 Statement of the main result
The main result establishes a stochastic maximum principle with first and second adjoint processes under stated assumptions. It applies to controlled parabolic SPDEs, including reaction-diffusion equations with a Laplacian operator.
3.1 Statement of the stochastic maximum principle
- The theorem assumes a complete orthonormal basis of L2 that is also a Schauder basis of L4, and an analytic L4 semigroup whose generator domain is compactly embedded in L4.The paper notes that the basis condition holds for a large class of measure spaces, typically with a Haar-type basis.Source: §3.1 · p. 5
- The second adjoint takes values in the bounded operators from L4 to L4/3, identified with bounded bilinear forms on L4.The associated operator space is endowed with the operator norm.Source: §3.1 · p. 5
- For an optimal pair, Theorem 3.3 provides progressive first and second adjoint processes satisfying the stated inequality for every control action.The first and second adjoints are (p, q1, . . . , qd) and P, respectively.Source: §3.1 · p. 5
3.2 Application to stochastic PDEs of parabolic type
- The result applies to the reaction-diffusion example with A = ∆ under the stated coefficient assumptions and an initial state in L^p for some p > 8.The paper verifies the required semigroup and basis hypotheses for this example, and notes that suitable elliptic operators with appropriate boundary conditions can also be considered.Source: §3.2 · p. 6
4 The spike variation method and the first adjoint process
The section develops spike variations and first-order adjoints to expand the state and cost, then motivates and defines a second adjoint through a bilinear form on L4. This construction avoids requiring a solution to the difficult operator-valued BSDE.
4.1 Spike variation method and expansion of the state and the cost
- The spike-variation expansion constructs first and second state variations so the perturbed state and cost can be expressed up to a small remainder.The text describes the state difference as X − X^ε − Y^ε − Z^ε and uses the variations to express J(u^ε) − J(u) up to a small remainder.Source: §4.1 · p. 6
4.2 The first adjoint process
- The first adjoint process is defined as the solution of a backward stochastic PDE.Its mild form is described together with the adjoint semigroup generated by A*; duality relations are used in the proof.Source: §4.2 · pp. 12–13
4.3 Some formal computations and heuristics
- The second adjoint is motivated by formal computations, but finding a suitable solution to the operator-valued BSDE is described as very difficult.The authors therefore follow a different proof strategy.Source: §4.3 · p. 14
- The second adjoint P_t is defined by a continuous bilinear form on L4, without reference to the operator-valued BSDE, and the desired identity is proved for this definition.The bilinear form is equivalently a bounded linear operator from L4 to its dual L4/3.Source: §4.3 · p. 15
5 End of the proof of the stochastic maximum principle
This section completes the proof of the stochastic maximum principle by constructing a second adjoint process valued in bounded operators on L4 and establishing regularity for the associated linearized stochastic equation. It develops estimates and uses compactness and finite approximations, then lets the approximation radius tend to zero to obtain (5.23).
5 End of the proof of the stochastic maximum principle
- The linearized stochastic equation has a unique mild solution with continuous L4 trajectories, and its estimates express regularization of initial data into L4.The stated regularization is described as sending data in D(−A)^−η to L4; the section uses the estimate ∥(−A)^ηe^tAf∥4 ≤ Cηt^−η and compact embedding for sufficiently small η, without pursuing possible weakenings of Hypothesis 3.2.Source: §5 · pp. 15–16
- A progressive operator-valued process P is constructed as the second adjoint process, with additional estimates for inputs in fractional domains D(−A)^η.The construction is used to establish the representation needed to conclude Theorem 3.3.Source: §5 · pp. 17–19
5.1 Proof of (5.17)
- Compact embedding of fractional-domain balls into L4 supports finite coverings and approximation arguments used to prove (5.17).The argument controls the relevant approximation on a compact subset of L4 and concludes convergence in the proof of (5.17).Source: §5.1 · p. 21
5.2 Proof of (5.18)
- Finite-dimensional projections and Hilbert-space Itô calculations are used to approximate the second-adjoint expression in the proof of (5.18).The proof passes through Yosida approximations and then takes limits in the finite-dimensional approximation; the supplied passage ends before the remaining estimate is shown.Source: §5.2 · pp. 22–24
- Compact embedding makes K a compact bounded subset of L4, so finite covers with centers in the dense space D(−A)^η approximate every f in K.The proof uses these approximations to derive its subsequent bounds.Source: §5.2 · p. 24
- With a bound independent of ε and δ, letting δ tend to zero makes the left-hand side vanish and yields (5.23).The preceding estimates are combined with expectation and (5.8) before taking this limit.Source: §5.2 · p. 24
A Stochastic integrals in Lp spaces
The appendix sketches stochastic integration in Lp spaces for integrands driven by a finite-dimensional Wiener process. It collects the construction and estimates used in the paper, including an extension to measurable integrands and a conditional variant.
A Stochastic integrals in Lp spaces
- The appendix collects the stochastic-integral facts in Lp spaces that are needed for the paper.The integrands take values over D and are measurable with respect to the progressive and spatial σ-algebras.Source: A Stochastic integrals in Lp spaces · p. 25
- For integrands of the specified special type, the stochastic integral is constructed pointwise using the finite-dimensional Wiener process, with estimates obtained from Burkholder-Davis-Gundy inequalities.The integral is then estimated in Lp by integrating over the spatial measure.Source: A Stochastic integrals in Lp spaces · p. 25
- The integral and its inequality extend to P ⊗ D-measurable integrands when the right-hand side of (A.1) is finite.A conditional variant is also stated under the existence of regular conditional probabilities, and the appendix notes its use in the proof of Proposition 5.2.Source: A Stochastic integrals in Lp spaces · pp. 25–26