Predefined-Time Resilient Integral Reinforcement Learning for Input-Constrained Unknown Nonlinear Systems Under FDI Attacks and Disturbances: A Fully Data-Driven Approach

I. INTRODUCTION

• Unknown drift, actuator limits, FDI attacks, and disturbances complicate nonlinear optimal control, while standard RL does not provide hard bounds or assignable settling times.The HJI equation generally depends on the unknown drift, and quadratic input penalties do not enforce hard actuator bounds.Source: §I · p. 1
• Prior IRL removes unknown drift from learning and can enforce input constraints, but existing approaches mainly guarantee asymptotic or ultimate convergence.Concurrent and composite learning use current and recorded information instead of sustained persistent excitation; predefined-time Bellman–Isaacs learning remains largely unexplored.Source: §I · p. 1
• Earlier predefined-time RL methods do not jointly provide finite-window Bellman–Isaacs learning, drift removal, actuator constraints, disturbances, attacks, and an assignable closed-loop deadline.The paper identifies a remaining timing issue: pre-convergence data stay in the integral residual until one complete reinforcement window elapses.Source: §I · p. 1
• A constrained minimax cost uses state penalties for predefined-time dissipation and a nonquadratic input term that preserves actuator bounds by construction.The framework derives a finite-window Bellman–Isaacs equation without unknown drift and computes the critic gain from an assigned deadline.Source: §I · p. 1
• Current integral information and finite recorded data replace persistent excitation after sufficient data are collected, supporting exact or practical critic convergence under attacks and disturbances.The coupled deadline accounts for data informativity, critic learning, reinforcement-window length, and state convergence; validation uses a two-link robot manipulator.Source: §I · p. 1

A. Preliminaries

• The normalized comparison theorem assumes positive parameters with 0 < p < q and pr < 1 < qr, and gives a predefined-time stability result under a derivative condition.The supplied theorem passages do not include the full derivative inequality.Source: A. Preliminaries · p. 2
• The assigned time Ts sets the settling-time scale, while the ratio α/β shapes the transition between the low- and high-V dissipation regimes.The remark states that replacing Ts by ηTs scales the required dissipation rate by 1/η.Source: A. Preliminaries · p. 2
• Finite L2-gain bounds the regulated output's energy by the exogenous input's energy and an initial-state term, with γ quantifying worst-case energy amplification.A storage-function dissipation inequality connects this attenuation condition to the corresponding HJI equation.Source: A. Preliminaries · p. 2

B. Secure Saturation-Constrained Differential Game

• The model treats the FDI signal as an additive matched input-channel attack applied after the controller generates its constrained secure action.The constraint applies to u only; the combined input u+a is not assumed to satisfy the same bound.Source: B. Secure Saturation-Constrained… · p. 2
• Attacks and disturbances are assumed bounded in both L2 and L∞, with known finite pointwise bounds.The operating domain is compact, and the plant admits a unique forward solution at least until the first exit from that domain.Source: B. Secure Saturation-Constrained… · p. 2
• A barrier-type penalty is positive definite and strictly convex on the admissible input set, with an unbounded gradient at its boundary.Consequently, the stationary minimizing policy remains strictly inside the set without post hoc clipping.Source: B. Secure Saturation-Constrained… · pp. 2–3
• The running state penalty is to be recovered by inverse-optimal construction so that the HJI identity generates the desired predefined-time dissipation.The formulation presumes a positive-definite saddle value function with stated regularity on the operating domain.Source: B. Secure Saturation-Constrained… · p. 3

C. Normalized Predefined-Time Comparison

• The comparison is applied outside a prescribed radius around the origin; the origin remains in the HJI domain and trajectories outside the terminal neighborhood can be compared again.The excluded region is analytical and does not alter the HJI operating domain.Source: C. Normalized Predefined-Time Comparison · p. 3
• The fixed-time state penalty combines quadratic, low-power, and high-power terms, with powers chosen on opposite sides of one.The penalty depends on measurable state rather than the unknown value function, and the HJI-shaped cost directly generates the two fixed-time powers.Source: C. Normalized Predefined-Time Comparison · p. 3
• A certified admissible policy supplies a computable worst-case performance bound that substitutes for the unknown value-function bound in the certificate.The prescribed operating domain and terminal radius are set from design specifications, physical limits, safety needs, or the intended data-collection region.Source: C. Normalized Predefined-Time Comparison · p. 3
• A small prescribed radius or large operating domain can make the certificate conservative, increasing required gains and potentially producing aggressive transients, stiffness, and tighter digital implementation demands.This is the stated tradeoff for an a-priori computable convergence certificate without knowing the exact value function.Source: C. Normalized Predefined-Time Comparison · p. 4

III. INTEGRAL BELLMAN–ISAACS LEARNING

• The ideal and learned value functions use basis-weight representations, with an approximation error in the ideal representation.The learned estimate is represented by the current critic weights and basis vector.Source: §III · p. 4
• The finite-window Bellman–Isaacs identity uses measured state transitions rather than direct drift evaluation, with trajectory mismatch entering the critic regression as an aggregate residual.The mismatch vanishes when the measured control and disturbance signals match the ideal saddle policies over the window.Source: §III · p. 4
• The normalized online and recorded residuals decompose into critic-weight error and bounded residual components.The online residual bound includes approximation error, running-cost mismatch, and finite-window trajectory mismatch.Source: §III · p. 5
• The reinforcement-window length trades residual accuracy against replay-stack conditioning: shrinking it reduces accumulated policy mismatch but can weaken numerical informativity.As the window shrinks, normalized replay regressors can approach zero, increasing the critic gain needed for a prescribed horizon.Source: §III · p. 5
• The critic loss uses residual powers below and above one to establish the two-power dissipation needed for fixed- and predefined-time convergence.The lower-power term dominates locally, while the higher-power term prevents the settling-time bound from growing with large initial weight error.Source: §III · p. 5
• Current integral information supplies new trajectory data, while a sufficiently rich finite replay stack preserves informative critic directions without continued probing.Once the replay stack meets its data-informativity condition, the probing signal may be removed.Source: §III · pp. 5–6

IV. PREDEFINED-TIME INTEGRAL CRITIC

• An exact two-power settling-time integral yields a less conservative deadline parameterization than conventional estimates that split the Lyapunov domain.The result evaluates the complete settling-time integral.Source: §IV · p. 6
• The critic gain can be selected from the assigned horizon because the settling-time scale varies inversely with the gain and is independent of initial error.The horizon is measured from the data-informativity instant.Source: §IV · p. 6
• A sufficiently rich replay stack converts stored critic regressors into coercive bounds on weight error, and the replay mechanism can be replaced or augmented by qualifying ICL or composite-learning constructions.The relevant alternatives require their corresponding finite-data richness conditions.Source: §IV · p. 6
• By the assigned horizon after data informativity, the critic error reaches zero with exact residuals or enters a forward-invariant set with bounded nonzero residuals.The nonzero-residual set is specified by a unique nonnegative threshold.Source: §IV · p. 7
• A designed finite excitation interval can separate the data-acquisition budget from critic convergence, without requiring continuous persistent excitation after replay informativity.If informativity is guaranteed within a known horizon, the critic horizon can be allocated from the overall deadline.Source: §IV · p. 8

• The proof combines critic-error dissipation with a moving-window state-energy estimate into a two-power composite Lyapunov inequality, using a derivative bound to prevent narrow energy concentration.Without derivative information, fixed window energy alone cannot provide the required positive lower estimate for the low-order state term.Source: §V · pp. 8–10
• Under Theorem 3’s hypotheses, the augmented state–critic signal enters a forward-invariant practical residual set by Tp, while learned inputs remain strictly within actuator bounds for all t ≥ 0.The residual set’s radius depends on the HJI residual, value-gradient approximation error, attack and disturbance reconstruction mismatch, and post-learning critic radius.Source: §V · pp. 10–12
• A single critic gain must satisfy the critic-learning horizon requirement and both closed-loop dissipation requirements simultaneously.The gain is bounded below by max{αW, αX,1, αX,2}.Source: §V · p. 10
• When conditions (111) and (112) hold, critic error vanishes after TE + TW; for t ≥ Tp, the state lies in Br−(0), establishing a practical-region bound, not exact convergence.Source: §V · p. 11
• The certified deadline splits as Tp = TE + TW + T + TX, a conservative worst-case budget rather than a required sequence because state and critic evolve simultaneously.TE builds an informative replay stack, TW is the critic-adaptation horizon, T flushes pre-learning data from the moving window, and TX is the convergence horizon.Source: §V · pp. 11–12
• The analytical term i leaves the HJI problem and nominal coordination objective unchanged, and predefined-time comparison applies again if disturbances or attacks drive xi outside Bri,−(0).Source: §V · p. 12

A. Simulation Setup

• The plant is a two-link planar manipulator with state x = col{q1, q2, ˙q1, ˙q2} and dynamics including actuator input, an attack, and external torque disturbance.The setup specifies its inertia, Coriolis, damping, and gravity terms.Source: A. Simulation Setup · p. 12
• The game uses Qx = diag(7, 6, 1.5, 1.2), R = 0.06I2, T = 0.8I2, S = I4, and actuator bound ¯u = 8.The setup also gives attack and disturbance attenuation parameters and predefined-time design parameters.Source: A. Simulation Setup · p. 12
• The critic uses 20 Gaussian radial basis functions and a replay stack of 200 samples, with informativity and conditioning requirements assessed numerically.For deadlines Tp = 2.2 s and Tp = 10 s, the setup assigns learning and state-convergence horizons and reports the resulting gains.Source: A. Simulation Setup · p. 12
• The tests specify a decaying matched FDI attack, a decaying external torque disturbance, and a nominal initial state x(0) = col{1.470, −1.155, 0.420, −0.315}.The multi-initial-condition test uses five initial-state norm levels from approximately 0.55 to 2.40.Source: A. Simulation Setup · p. 12
• Predefined-time IRL is compared with Fixed-time IRL and conventional ADP-IRL using the same replay data and learning interval.The initial critic weights are selected to preserve an admissible stabilizing policy before learning improves the critic.Source: A. Simulation Setup · p. 12

B. Simulation Results

• Replay data reach the required finite-data level 3 × 10^-2 at approximately 0.13 s, before the 0.30 s acquisition interval ends.The smallest-singular-value indicator later increases to approximately 1.24 × 10^-1.Source: B. Simulation Results · p. 12
• Most critic-weight variation occurs within 5 × 10^-2 s, after which both groups are practically stationary well before TW; both deadlines yield essentially identical weight solutions (Fig. 1).The reported fast transient is attributed to informative replay data, normalization, two-power learning, and the deadline-dependent gain.Source: B. Simulation Results · p. 12, Figure 1 · p. 12
• At Tp = 2.2 s, Predefined-time IRL drives states toward the origin much faster than at Tp = 10 s, when they approach it several seconds before the deadline (Fig. 2).The regulation objective is unchanged; the available time differs.Source: B. Simulation Results · p. 12, Figure 2 · p. 13
• The matched FDI attack and external torque disturbance remain nonzero around the Tp = 2.2 s deadline, so the fast response is not obtained by removing adverse signals (Fig. 4).Source: B. Simulation Results · p. 13, Figure 4 · p. 13
• For Tp = 10 s, all five initial conditions with norms 0.55–2.40 enter rx = 0.04 with a large time margin; at Tp = 2.2 s, the farthest enters shortly after the deadline (Fig. 5).The closest initial conditions enter within or near the shorter deadline.Source: B. Simulation Results · p. 13, Figure 5 · p. 13
• At Tp = 2.2 s, Predefined-time IRL reaches state norm of order 10^-2 below rx = 0.04, while Fixed-time IRL and ADP IRL remain approximately one order larger (Fig. 6).At Tp = 10 s, all three methods approach a small neighborhood of the origin, and their differences are less significant.Source: B. Simulation Results · p. 13, Figure 6 · p. 13
• At Tp = 2.2 s, Predefined-time IRL nearly uses the admissible magnitude ū = 8, versus peaks of approximately 6–6.5 for the other methods; the shorter settling costs greater control effort (Fig. 7).At Tp = 10 s, strong actuation is not necessary.Source: B. Simulation Results · p. 14, Figure 7 · p. 14
• Finite control authority prevents arbitrarily reducing the prescribed time; establishing an explicit lower bound on feasible deadlines is identified as future work.The feasible time depends on actuator limits, system parameters, the initial-condition region, and disturbance or attack bounds.Source: B. Simulation Results · p. 14

VII. DISCUSSION

• Distributed predefined-time IRL is proposed for graph-coupled systems with heterogeneity, partial measurements, delays, switching information structures, and network growth.The direction includes explicit network-dependent constants and a common prescribed deadline.Source: §VII · p. 14
• A networked extension should jointly address learning, control, communication scheduling, security, and channel capacity under a wall-clock deadline.The discussion identifies trade-offs among the deadline, communication or bit rate, packet loss, denial-of-service severity, quantization, and asynchronous updates.Source: §VII · p. 14
• Predefined-time tracking and safety learning could combine reachability guarantees with convergence deadlines, while Koopman warm starts could supply an initial predictor, stabilizing policy, and value approximation.The proposed warm-start direction emphasizes certifying regional admissibility, propagating lifting and finite-data errors, and including construction time in the deadline.Source: §VII · p. 14
• The present theory assumes sufficient value-function smoothness; viscosity sub/supersolution arguments are proposed for extending prescribed-time analysis to nonsmooth constraints and hybrid settings.The discussion names safety boundaries, switching communication modes, and hybrid dynamics as settings for the proposed extension.Source: §VII · p. 14

VIII. CONCLUSION

• The proposed critic-only IRL framework addresses unknown drift dynamics alongside input constraints, FDI attacks, and external disturbances.Finite-window Bellman–Isaacs identities, finite replay data, and a two-power critic update form the approach.Source: §VIII · p. 14
• The critic and closed-loop state are driven into practical regions within designer-assigned times, with gains determined explicitly from the prescribed deadline.The Lyapunov analysis accounts for data-acquisition time and learning-window delay, and the saturated policy preserves actuator constraints by construction.Source: §VIII · pp. 14–15
• The simulations show that convergence time can be actively shaped, but at the cost of increased control effort and within finite actuator capability.Source: §VIII · p. 15

APPENDIX A PROOF OF THEOREM 1

• The proof uses a transformed Lyapunov-function argument to establish finite settling and arrival at the origin.It bounds the settling time through an integral inequality and uses positive definiteness to conclude that reaching zero Lyapunov value means the state is zero.Source: Appendix A · p. 15

APPENDIX B DATA-DRIVEN CONSTRUCTION OF AN ADMISSIBLE INITIAL POLICY FOR DEFINITION 2 AND ASSUMPTION 3

• A finite-data Koopman predictor can construct an admissible stabilizing warm-start policy when the stated stability, error, and input-constraint conditions hold.This policy safeguards the state in the compact operating set; the predefined-time ADP–IRL mechanism supplies optimality and the convergence deadline.Source: Appendix B · p. 15

APPENDIX C PROOF LEMMA 7

• The Appendix C proof bounds a window maximum by splitting into two cases and combining the resulting estimates.Source: Appendix C · p. 15