A Tutorial on Physics Constrained Machine Learning for Hybrid Modeling, Optimization, and Control

1 Introduction

• Purely data-driven models can suffer from scarce data, poor extrapolation, and physically meaningless predictions, while incorporating domain knowledge can improve data efficiency, robustness, and generalization.Source: §1 · p. 2
• Penalty-based training does not generally guarantee exact feasibility or stationarity, and soft constrained neural networks may therefore fail to satisfy physical constraints.Source: §1 · pp. 2–3
• Hard-constrained networks enforce constraints by design; projection-based methods restore feasibility by projecting predictions onto a constrained manifold, including for general nonlinear constraints.Source: §1 · p. 3
• Projection separates feasibility restoration from loss minimization, and training across parameter realizations may help networks approximate active-set transitions for repeated optimization problems.Such predictions can serve as warm starts for conventional solvers, while hard-constrained learning is presented as a low-inference-cost option for repeated parametric problems.Source: §1 · pp. 3–4
• KKT-HardNet combines a neural-network backbone with projection-based constraint enforcement, and the tutorial adds projection strategies, implicit differentiation, sparse computation, and ahead-of-time compilation.The tutorial covers hybrid modeling, process simulation, optimization, and model predictive control.Source: §1 · p. 4

2.1 Neural Network Backbone

• Neural networks map input features through sequential layers to predicted outputs, with the final layer generally omitting an activation function.The framework accepts different neural-network architectures as backbones.Source: §2.1 · pp. 4, 6, §2 · p. 4

2 KKT-HardNet: A Physics Constrained Machine Learning Framework

• KKT-HardNet projects a neural network’s raw prediction onto the feasible set, producing an output that satisfies the physical constraints.The framework combines a neural-network backbone, projection layer, and loss function.Source: §2 · p. 4, §2.2 · p. 7

2.2 Projection Layer

• For nonlinear constraints, the projection layer can use KKT-based iterative solves and implicit differentiation to compute training gradients.The projection problem finds a minimum-distance feasible output; implicit differentiation applies when the converged solution has a locally differentiable implicit representation.Source: §2.2 · pp. 6–7

2.3 Loss Function and Backpropagation

• Training minimizes a task-specific loss on projected predictions, using data discrepancy for supervised modeling or the physical or economic objective for optimization and control.The objective may include a regularization term.Source: §2.3 · p. 7

2.4 A Modular Python Package (pip install kkt-hardnet)

• The package offers a modular workflow for defining constraints, selecting backbones and losses, training models, and exporting predictors.Projection options include explicit projection for linear equalities, KKT solves with Fischer–Burmeister reformulation, and a primal-dual interior-point method; implicit function theorem is used for backward gradients.Source: §2.4 · pp. 8–9

3 PCML for Hybrid Modeling

• Hybrid modeling combines process data and partial first-principles knowledge to predict complex systems whose mechanistic models may be difficult or costly to solve.Neural-network predictions alone are not guaranteed to satisfy known physical relations.Source: §3 · p. 11
• The projection layer enforces incorporated physical relations while data-driven learning supplies relationships not included in the constraints.The complete governing model is not required; only available physical knowledge is included in the projection layer.Source: §3 · p. 11

3.1.1 A case study on chemical process simulation involving nonlinear thermodynamics

• The chemical-process case studies compare numerical and computational performance with the base study reported by Iftakher et al..Table 2 covers performance comparison for the extractive-distillation column case study.Source: §3.1 · p. 11, Table 2 · p. 12

3.1.2 A case study on pooling problem involving nonlinear equality and inequality constraints

• The pooling case study has four inputs, five outputs, four equality constraints, and two inequality constraints, including nonlinearities.Its process flowsheet forms the product from three feeds.Source: §3.1.2 · p. 12, Figure 7 · p. 12

3.2 Hybrid Modeling of a Two-Phase Reactor-Condensor System

• For the reactor–condenser model, training data came from steady-state DAE solutions over six Latin-hypercube-sampled process inputs.The projection layer uses thermodynamic equalities and inequality constraints, while steady-state conditions are not imposed as projection constraints.Source: §3.2 · pp. 13–14
• KKT-HardNet achieved MSE of 3.11×10−3, maximum equality and inequality violations of 8.59 × 10−7 and 9.45 × 10−7, and 786.21 s training and 57.86 ms inference; ENFORCE was slower but tighter.ENFORCE required 16362.92 s for training and 208.39 ms per inference batch; the unconstrained MLP and soft-constrained PINN were cheaper but had larger physical-constraint violations.Source: Table 5 · p. 16, §3.2 · p. 16

4 PCML for Real-Time Optimization

• KKT-HardNet learns the parametric solution map offline, using a loss to favor economically beneficial solutions and projection to enforce constraints during inference.This approach maps changing process or economic parameters directly to feasible near-optimal operating conditions rather than repeatedly solving the original problem online.Source: §4 · p. 16

4.1 A CSTR Optimization Case Study

• KKT-HardNet reproduced the reference optimizer objective value of 0.535, with very small equality residuals, zero inequality violation, and 1.65 ms inference versus 1.83 ms for the reference optimizer per batch of 100.PINN obtained a lower objective while violating both constraint types, whereas DC3 remained feasible but missed the optimal objective value.Source: Table 7 · p. 19, §4.1 · p. 19
• As heat price increased, optimal heating fell from approximately 5.62 × 10^4 to 1.33 × 10^4 cal/s, and optimal reactor temperature fell from 436.27 to 410.34 K.The optimal objective increased from 0.497 to 0.560; C_B decreased from 0.509 to 0.467 mol/L, reflecting the stated conversion–energy-cost trade-off.Source: §4.1 · p. 19
• Across 500 warm single-instance predictions, mean and median inference latencies were 38.34 and 38.32 ms, with a 66.49 ms maximum, versus a 1000 s RTO update interval.The first evaluation took 7.67 s because of one-time JAX compilation and was excluded from online timing statistics.Source: §4.1 · p. 19

5 PCML-based Model Predictive Control

• KKT-HardNet learns the parametric MPC solution map offline, projects predictions onto the constraints, and implements only the first control action before repeating with the updated state.The approach targets the repeated computational cost of nonlinear MPC at each sampling instance.Source: §5 · p. 20

5.1 A CSTR Control Example

• The CSTR MPC uses current concentrations and temperature with set points to optimize heat input over a finite horizon, subject to process dynamics, operating bounds, move limits, and a Lyapunov condition.The first optimized control action is applied, and the process is propagated before optimization repeats.Source: §5.1 · pp. 20–21
• Across 2000 controller evaluations, mean and maximum equality residuals were 1.90 × 10−9 and 5.15 × 10−7, and no state, heat-input, control-move, or plant-bound violations occurred.Mean and maximum inequality violations were 7.72 × 10−13 and 2.95 × 10−9; the remaining residuals came from the Lyapunov condition and stayed below the 10−8 projection tolerance.Source: §5.1 · p. 24
• The closed-loop controller tracked RTO set points with RMSEs of 6.35 × 10−4 mol/L for C_A, 6.34 × 10−4 mol/L for C_B, and 0.278 K for temperature.Settling took approximately 185–255 s after RTO updates, with maximum overshoot below 0.03% for all controlled variables.Source: §5.1 · p. 24, Figure 13 · p. 25, Table 9 · p. 25
• The Lyapunov function rose after each RTO set-point update and then decayed toward zero, while its condition showed no violations above the 10−8 tolerance across 2000 evaluations.The minimum Lyapunov margin was −2.95 × 10−9 and the mean was 1.19 × 10−5.Source: Figure 14 · p. 26, §5.1 · p. 26
• 20.20 ms mean and 19.94 ms median single-instance inference latencies used only 1.59% of the 5 s sampling interval at maximum, with no deadline misses.Combined RTO–MPC median and maximum latencies were 50.95 and 59.48 ms at the nine sequential evaluations.Source: §5.1 · p. 26
• Over the 10,000 s simulation, mean plant and RTO objectives were 0.54018 and 0.54025, with a mean absolute economic gap of 2.12×10−4.The maximum absolute gap was 4.50×10−3 immediately after an RTO update, after which MPC returned the plant objective toward the RTO target.Source: §5.1 · p. 26, Figure 15 · p. 27

6 PCML for Grey-Box Optimization

• KKT-HardNet’s differentiable surrogate propagates downstream objective gradients to operating conditions while keeping predictions consistent with incorporated physical constraints.The paper notes that surrogate inaccuracy or physical inconsistency can lead an optimizer to exploit prediction errors.Source: §6 · p. 27

6.1 Simulation-based Optimization of a Syngas Production Process

• The syngas model predicts five outlet component flows and total utility cost from feed ratio, operating pressure, and temperature, enforcing carbon, oxygen, and hydrogen balances.The process data came from 4000 Aspen Plus simulations sampled over the three operating conditions.Source: §6.1 · pp. 28–29
• KKT-HardNet’s syngas surrogate achieved an MSE of 4.48×10−5 and mean and maximum elemental-balance residuals of 2.25 × 10−16 and 3.55 × 10−15, with the lowest training time among hard-constrained approaches.Its inference time was higher than the unconstrained and soft-constrained neural-network baselines.Source: Table 10 · p. 30, §6.1 · p. 30
• The syngas optimization varies feed ratio, operating pressure, and temperature using a scalarized objective that trades utility consumption against CO2 conversion.The study solves for α values from 0 to 1 using 100 random initializations per value, with operating variables restricted to prescribed bounds.Source: §6.1 · p. 30
• For α≈0.06–0.74, optimization repeatedly selected a≈4.79, P_op = 1 atm, and T_op = 100 °C, with predicted CO2 conversion near 0.999 and utility approximately 0.345.For α≳0.86, the optimum reached a=1, P_op = 1 atm, and T_op = 100 °C, with utility approximately 0.110 and conversion approximately 0.257.Source: §6.1 · p. 30, Figure 18 · p. 32

7 Conclusions

• KKT-HardNet supports custom neural-network backbones and projection algorithms, and implicit differentiation trains through iterative projection layers without unrolling solver iterations.The framework is demonstrated across process simulation, real-time optimization and control, and data-driven optimization.Source: §7 · p. 32
• The paper identifies larger-scale applications and problem-specific projection methods that exploit sparsity and structure as future work.It aims to establish KKT-HardNet as a practical development and benchmarking platform for physics-constrained machine learning.Source: §7 · p. 32

Declaration of Generative AI and AI-Assisted Technologies

• Generative AI tools assisted with initial code, language editing, grammar, readability, and clarity, but not research data, scientific analyses, conclusions, or scientific claims.The authors state that all AI-assisted content was reviewed, tested, and modified, and that they take responsibility for the manuscript’s scientific content and integrity.Source: Declaration of Generative AI and… · p. 32

• Data loss is measured by mean squared error, while optimality measures distance from the true optimum in optimization case studies.Source: A Benchmarking: metrics and other… · p. 36
• Feasibility uses absolute equality residuals and positive inequality residuals, reported as both maximum and mean violations.Source: A Benchmarking: metrics and other… · p. 36
• Inference speed is measured per batch after training, using batches of 100 instances across case studies.Source: A Benchmarking: metrics and other… · p. 36
• Comparators include IPOPT, MLP, PINNs, ENFORCE, and DC3, with the stated implementations or default codespace settings used for ENFORCE and DC3.Source: A Benchmarking: metrics and other… · p. 36
• The case studies reuse library imports, backbone setup, training configurations, and solver configurations.Source: A Benchmarking: metrics and other… · p. 36

B.1 Explicit Projection: Linear Equality Constraints

• For linear equality constraints, the projection has an explicit affine solution whose matrices can be precomputed and reused for a single-iteration feasibility projection.Source: §B.1 · pp. 36–37

B.2 Implicit Projection I: KKT conditions via Fischer-Burmeister Reformulation

• The Fischer–Burmeister formulation encodes dual and complementarity conditions without loss of fidelity, alongside stationarity and primal feasibility equations.The resulting nonlinear KKT system is solved iteratively with Gauss–Newton and Armijo backtracking; for linear equalities it recovers the explicit solution.Source: §B.2 · p. 37

B.3 Implicit Projection II: Primal-Dual Interior Point Method

• The primal-dual interior-point method is implemented in a primitive form suited to batch inference and GPU runs.Source: §B.3 · p. 37

C Reduction and Heuristics on KKT Conditions via Fishcer-Burmeister Reformulation

• The reduced Fischer–Burmeister approach solves a smaller symmetric system, using diagonal pivoting and regularization when needed.The reduction acts only on slack variables, so performance changes are visible only when inequalities are present.Source: C Reduction and Heuristics on KKT… · pp. 37–38

• The mean absolute LMPC correction was 0.736 cal/s, with RMS 2.14 cal/s and maximum magnitude 13.81 cal/s, against nominal heating rates of approximately 2.34 × 10^4 to 3.96 × 10^4 cal/s.Source: §D.1 · p. 38
• The applied heat-input variation was 2.34 × 10^4 cal/s versus 2.32 × 10^4 cal/s for the RTO target, an increase of only 198.8 cal/s or approximately 0.86%.The largest change between successive implemented inputs was 3.26 × 10^3 cal/s, and the input stayed within physical bounds.Source: §D.1 · p. 38

D.2 Closed-Loop Performance under Challenging Heat-Price Profiles

• All tested price profiles remained stable and within plant bounds; temperature RMSE ranged from 0.278 K for baseline to 1.85 K for random prices.The spike and random profiles produced maximum temperature errors of approximately 22.05 K and 20.22 K, respectively, while the ramp had essentially zero positive economic gap.Source: §D.2 · pp. 38–39, Table 11 · p. 39, Figure 20 · p. 40
• Applied heat input, tracking errors, LMPC correction, and positive economic gap are shown across baseline, sinusoidal, random, spike, and ramp price trajectories.The correction is largest after RTO set-point updates and then decays toward zero.Source: Figure 19 · p. 39, Figure 20 · p. 40

E Multi-Start Projected Adam Optimization

• Syngas optimization varies feed-flow ratio, reactor pressure, and temperature within normalized training-data bounds, using projected Adam updates on a scalarized utility–conversion objective.Projection keeps iterates in the prescribed operating domain.Source: E Multi-Start Projected Adam… · pp. 40–41
• For each scalarization weight, the method repeats optimization from random starts and retains the lowest-objective solution, reducing dependence on initialization without guaranteeing a global optimum.Source: E Multi-Start Projected Adam… · p. 41