Linearly constrained Gaussian processes

• A transformation of an underlying function incorporates known linear constraints into Gaussian processes, guaranteeing that samples and predictions satisfy them.Source: Abstract · p. 2
• The paper proposes a constructive procedure for designing the transformation operator.Source: Abstract · p. 2
• The method is illustrated with simulated and real-data examples.Source: Abstract · p. 2

• Gaussian processes combine flexible model structures with principled ways to encode prior knowledge, and their utility spans many real-world applications.Source: §1 · p. 2
• Adding fictitious constraint measurements is straightforward, but increases problem dimension and does not enforce constraints between sampled points.Source: §1 · p. 3
• Existing constraint-obeying covariance approaches enforce constraints everywhere without increasing problem dimension, but their scaling to arbitrary linear operators is unclear.Source: §1 · p. 3
• The paper transforms a given GP into a derived GP that satisfies constraints, with an algorithm for constructing the transformation and demonstrations on simulated and magnetic-field data.Source: §1 · p. 3
• The introduction motivates the construction with a two-dimensional function required to obey a differential equation.Source: §1 · p. 3

• The regression setup uses input-output observations and allows both inputs and outputs to be vector-valued, with zero-mean white noise representing measurement uncertainty.Source: §2 · p. 3
• Constraints are expressed through linear operators on the function, including linear transforms of outputs and partial differential equations.Source: §2 · p. 3
• The objective is to encode known constraints, arising from physical laws or other prior knowledge, in the GP mean and covariance so every prior sample obeys them.Source: §2 · p. 3

3.1 Approach based on artificial observations

• Artificial observations encode zero-valued constraints at selected inputs, where the regression treats them as noise-free observations.The approach can also represent constraints with a non-zero right-hand side, and has no theoretical limit on the number of constraints.Source: §3.1 · p. 4
• Increasing artificial constraint observations raises memory and runtime costs and worsens numerical stability, while enforcement remains pointwise.Adding more points can address pointwise coverage but exacerbates the computational and conditioning problems.Source: §3.1 · p. 4

3.2 A new construction

• Represent the target as f(x) = G_x[g], choose a GP prior for g, and derive the mean and covariance for f using GP closure under linear operations.The construction proceeds by finding G_x, choosing g's mean and covariance, and deriving f's mean and covariance.Source: §3.2 · pp. 4–5
• The construction separates constraint encoding from other kernel properties, preserves the problem size, and guarantees a valid covariance when the underlying covariance is valid.It also provides a straightforward construction for complicated constraints, whether component relations are explicit or implicit.Source: §3.1 · p. 4, §3.2 · p. 5

3.3 Illustrating example

• For a two-dimensional divergence-free vector field, the constructed covariance structure guarantees that every generated function fulfills the constraint.The example uses a scalar underlying function to construct the target vector field.Source: §3.3 · p. 5

4 Finding the operator Gx

• The operator nullspace differs from the real-number algebra analogue; for the derivative constraint, it consists of constant functions.The derivative example motivates care when reasoning about operator nullspaces from real-number linear algebra.Source: §4 · p. 5
• The construction seeks Gx from a given Fx by proposing an operator-form ansatz, collecting coefficients into a homogeneous linear system, and using its nullspace to build Gx.If the nullspace is empty, the operator set in the ansatz must be extended and the procedure repeated.Source: §4 · pp. 5–6
• The algorithm parameterizes candidate columns of Gx, and may require iteration to find a suitable operator set.The authors note that alternatives to this parametric approach may be worth examining.Source: §4 · p. 6

4.1 Divergence-free example revisited

• In the divergence-free example, assuming continuous second derivatives yields a one-dimensional nullspace and recovers the solution previously found by inspection.The nullspace is spanned by [0, −1, 1, 0]T up to a scalar factor.Source: §4.1 · p. 6

4.2 Generalization

• The general algorithm leaves operators out of the solution process, making notation more cumbersome; the paper presents a simplified version and places the general algorithm in supplementary material.The authors describe the term-expansion procedure in the simplified presentation as somewhat informal.Source: §4.2 · p. 7

5.1 Simulated divergence-free function

• The simulation uses 50 noisy measurements and 400 prediction points; error is assessed by root mean squared error averaged over 50 reconstructions.Figure 2 shows the true and measured fields alongside three reconstructed-field error plots; artificial constraint observations coincide with prediction points.Source: §5.1 · p. 7, Figure 2 · p. 7
• The proposed approach is clearly better in simulation; artificial-observation error falls only to a point as observation count grows and the Gram matrix becomes larger and worse conditioned.Figure 3a reports the simulated comparison.Source: §5.1 · p. 7, Figure 3 · p. 8

5.2 Real data experiment

• The real-data experiment models a magnetic field as curl-free and averages errors over 50 reconstructions using 500 training and 1,000 test points.Training and test data are randomly selected from the collected dataset in each iteration.Source: §5.2 · pp. 7–8
• On real data, artificial observations approach the proposed method's accuracy at high counts, then accuracy declines at the final increase, probably because of ill-conditioning.Figure 3b reports the real-data comparison.Source: §5.2 · p. 8, Figure 3 · p. 8

6 Related work

• Prior work addresses inequality constraints,,, training-data linear constraints and quadratic forms, and anatomical correctness.Other work considers input ordering [23], medical applications,, circular variables [14], and one-dimensional PDEs [9, 15, 16].Source: §6 · p. 8
• Earlier approaches often enforce constraints externally through approximations or finite representations, so continuous satisfaction is not necessarily guaranteed.The paper cautions that methods developed for different constraint types cannot be directly compared.Source: §6 · p. 9
• Multivariate GP covariance methods include separable kernels, invariant kernels, and Taylor-expansion models based on known output relationships,.Curl- and divergence-free kernels inherit structure from physics and have improved regression accuracy.Source: §6 · p. 9
• The method uses linear transformations to build an invariant covariance for a multivariate GP, guaranteeing known linear constraints throughout the domain.The constraints are linear combinations of outputs known to equal zero; comparisons across different constraint types are not direct.Source: §6 · p. 9

7 Conclusion and future work

• The method guarantees that samples from the resulting multivariate Gaussian process obey known linear operator constraints at every point.Numerical simulations show benefits over alternative approaches, and the method also improved performance on real data.Source: §7 · p. 9
• The construction relies on expressing the target function through an underlying potential, whose existence is not mathematically guaranteed in the paper.The authors call for more rigorous requirements on Gx and a clearer relation between the potential and target function.Source: §7 · p. 9
• Extensions to nonlinear operators and constraints with a non-zero right-hand side are identified as future research topics.Source: §7 · p. 9

9.1 Linear operators

• A linear operator maps a function f(x) to g(z), and examples include differentiation, integration, and input wrapping, which can change the input dimension.Source: §9.1 · p. 10

9.2 Gaussian processes under linear operations

• Gaussian processes remain Gaussian under linear operations, with the operator applied to the mean and both covariance arguments; derivative use requires a differentiable covariance.Matérn1/2 is given as an example of a non-differentiable covariance.Source: §9.2 · p. 10

9.3 Generalization of Section 4

• The generalized construction finds transformations in the nullspace of operator equations, extending the parametric approach to arbitrary operators using homogeneous polynomials.The supplementary material treats first-order operators first, then operators of any order.Source: §9.3 · p. 11

9.3.1 First order operator equation

• For first-order operator equations, the method represents operators as a vector space and seeks vectors g satisfying Fg = 0 for all operator variables.The basis components can represent derivative operators.Source: §9.3.1 · p. 11

Example 1 (divergence free vector field)

• In the divergence-free example, the construction seeks a vector of operators whose product with the operator matrix is zero, then characterizes its nullspace.Source: Example 1 (divergence free vector field) · pp. 11–12

Example 2 (curl free vector field)

• For the curl-free vector-field example, the operator nullspace is spanned by a single basis vector, and the resulting covariance is used in the real-data experiment.The supplementary material gives the squared exponential covariance as an example.Source: Example 2 (curl free vector field) · pp. 12–13

9.3.2 Higher order operator equation

• For higher-order equations, operators are represented in homogeneous-polynomial spaces, producing algebraic conditions of order q + qg on the transformation coefficients.The formulation accounts for permutations of the operator-variable indices.Source: §9.3.2 · pp. 13–14

9.4.1 Experiment setup

• A Trivisio Colibri IMU sampled magnetometer, accelerometer, and gyroscope data at 100 Hz, while a Vicon system tracked five platform markers.A Google Nexus 5 was mounted for validation, but its data was not used.Source: §9.4.1 · p. 14

9.4.2 Experiment execution

• The platform was moved by hand through a 4 × 4 × 2 meter volume for 187 seconds, with sensor and optical-reference data collected asynchronously.Source: §9.4.2 · pp. 14–15

9.4.3 Pre-processing of data

• Angular-velocity correlation synchronized the optical and Trivisio streams; optical pose and sensor displacement yielded global positions, and magnetometer readings were rotated into global coordinates.The resulting position and magnetic-field data served as inputs and outputs, respectively, in Section 5.2.Source: §9.4.3 · p. 15