Resource-Efficient Distributed Recursive Gaussian Processes

I. Introduction

• GPs model unknown functions without a fixed functional form and quantify predictive uncertainty, making them useful for multi-agent estimation.Their applications include wireless traffic prediction, safe path planning, and cooperative mapping.Source: §I · p. 1
• Centralized processing requires agents to transmit measurements to a common unit, while standard GP regression scales cubically with observation count.Communication can consume a significant share of energy in UAV, multi-robot, and wireless-sensor systems.Source: §I · pp. 1–2
• ADMM-RGP and PDMM-RGP reduce communication relative to Consensus-RGP while maintaining comparable accuracy and network-wide consensus.PDMM-RGP is particularly effective on sparse communication networks.Source: §I · p. 2
• The authors analyze both algorithms’ stability, provide parameter-selection strategies for fast convergence, and evaluate them on a real-world multi-output wind dataset.They compare communication and computational complexity across graphs with varying connectivity.Source: §I · p. 2

A. Multi-Output Gaussian Processes

• The linear model of coregionalization represents each output as a linear combination of latent single-output functions, with a matrix-valued kernel capturing output correlations.The method’s inducing-point formulation is kernel agnostic and can extend to alternative GP representations such as random Fourier features.Source: A. Multi-Output Gaussian Processes · pp. 2–3, §I · p. 2

B. Recursive Gaussian Processes

• RGP updates a posterior over latent function values at fixed basis locations as new measurement batches arrive.The method uses a sparse inducing-point approximation and Kalman filter-like updates.Source: B. Recursive Gaussian Processes · p. 3

C. Distributed Recursive Gaussian Processes

• Each agent seeks the estimate that would result from jointly processing measurements across the network, but the global information updates require contributions from every agent.Agents therefore maintain local information vectors and matrices and exchange information with neighbors to approximate the global solution.Source: C. Distributed Recursive Gaussian… · p. 3

D. State of the Art: Consensus-RGP

• Consensus-RGP applies local RGP updates and then runs K rounds of distributed average consensus to approximate global information parameters.The algorithm extends RGP to distributed multi-output modeling.Source: D. State of the Art: Consensus-RGP · p. 3

A. ADMM-RGP

• ADMM-RGP adapts a communication-efficient distributed ADMM scheme and presents its updates directly in information form, simplifying convergence analysis.The equality constraint enforces consensus because the Laplacian’s nullspace is the consensus subspace.Source: A. ADMM-RGP · pp. 3–4
• The information-vector optimization uses local measurement contributions and prior information, with a Laplacian constraint coupling agents toward consensus.The updates use primal and dual variables, including a step size for dual ascent.Source: A. ADMM-RGP · p. 4
• For the information-matrix updates, ADMM applies half-vectorization, mapping the symmetric information matrix to a vector of dimension Dω = 1/2 Dξ(Dξ + 1).The algorithm performs local updates and computes each agent’s mean and covariance after K iterations.Source: A. ADMM-RGP · p. 4

B. PDMM-RGP

• PDMM applies to the node-separable ADMM-derived problems, whose objectives are strongly convex and differentiable with convex constraints.PDMM is reported to converge faster than ADMM in certain settings, including average consensus problems.Source: B. PDMM-RGP · p. 4
• The PDMM formulation assigns two directed edges to each communication link and imposes paired constraints on neighboring agents’ information vectors.A lifted constraint matrix and edge-indexed auxiliary dual variables organize the updates.Source: B. PDMM-RGP · pp. 4–5
• The PDMM iterations use parameter c > 0, and the information-vector update is obtained by solving an invertible system involving I + cCᵀC.Agents can broadcast their primal variables so neighbors reconstruct auxiliary values rather than receiving separate unicast transmissions.Source: B. PDMM-RGP · p. 5

• The section analyzes convergence properties of both proposed algorithms, ADMM-RGP and PDMM-RGP.Source: §IV · p. 5

A. ADMM-RGP Convergence

• For τ < 0, faster convergence than Consensus-RGP can be achieved while preserving complexity.Source: A. ADMM-RGP Convergence · p. 6
• The augmented Lagrangian remains convex in the primal variable when τ > −1/λN(L), supporting the stated negative-τ stability range.Source: A. ADMM-RGP Convergence · p. 6
• Under the stated parameter conditions, the ADMM-RGP error dynamics matrix is Schur stable and the algorithm converges.Source: A. ADMM-RGP Convergence · p. 6
• A grid search over the theorem’s parameter range can select α and τ to minimize ρ(M) for faster convergence.Source: A. ADMM-RGP Convergence · p. 6
• When only a small number K of rounds is used and transients dominate, parameters may instead minimize ∥M^K∥F.Source: A. ADMM-RGP Convergence · p. 6

B. PDMM-RGP Convergence

• The PDMM iterations converge for any c > 0 because the objectives of P1 and P2 are closed, strongly convex, proper, and differentiable.Source: B. PDMM-RGP Convergence · p. 6
• The auxiliary-variable dynamics are decomposed over invariant subspaces Ψ and Ψ⊥, with AΨ and AΨ⊥ restricting Aζ to each subspace.Source: B. PDMM-RGP Convergence · p. 6
• The non-convergent Ψ⊥ component does not prevent ξ[k] from converging, since the relevant error converges asymptotically to zero.Source: B. PDMM-RGP Convergence · p. 6
• Parameter c can be searched to minimize ρ(AΨ), or to minimize the transient-error norm when small K makes transient behavior dominant.Source: B. PDMM-RGP Convergence · p. 7

V. Experiments

• The experiments use 10-meter wind u- and v-components from the ERA5 post-processed daily statistics dataset.Source: §V · p. 7
• Ten agents collect 20 measurements each per time step over 50 steps, yielding 10,000 training data points.Source: §V · p. 7
• The simulations use two communication graphs, 400 basis locations, and 2,500 prediction locations to evaluate the trained model.Source: §V · p. 7, Figure 2 · p. 7

• The predictive distribution at test locations is expressed as a Gaussian with mean and covariance.Source: A. Performance Metrics · p. 7
• RMSE is used to evaluate the accuracy of predictions.Source: A. Performance Metrics · p. 7
• MVOP measures consensus using the variance of agents’ predictions around their sample mean.Source: A. Performance Metrics · p. 7

B. Results and Discussion

• All four methods achieved an average RMSE of 0.0862 across 100 Monte Carlo simulations, with a 95% confidence interval of [0.0860, 0.0864].Source: B. Results and Discussion · p. 7
• On graph 1, ADMM-RGP with K = 7 matched Consensus-RGP with K = 10 in consensus, reducing communication by 30% without sacrificing performance.Source: B. Results and Discussion · pp. 7–8, Figure 3 · p. 8
• On graph 2, PDMM-RGP achieved MVOP values up to three orders of magnitude lower than Consensus-RGP, and both proposed methods matched its K = 10 performance at K = 5, reducing communication by 50%.Source: B. Results and Discussion · p. 8, Figure 4 · p. 8
• Across communication graphs, PDMM-RGP performs best on sparse graphs, while ADMM-RGP consistently outperforms Consensus-RGP and performs particularly well on dense graphs.Source: B. Results and Discussion · p. 8, Figure 4 · p. 8
• All three algorithms scale linearly in computation time with neighbor count, but ADMM-RGP and PDMM-RGP can reduce total computation by requiring fewer rounds despite higher per-round complexity than Consensus-RGP.Source: B. Results and Discussion · p. 8, Figure 5 · p. 8

VI. Conclusion

• The conclusion reports fewer resource requirements with comparable reconstruction performance and identifies asynchronous communication and time-varying functions and graphs as future work.Source: §VI · p. 8

Appendix A Proof of Theorem 1

• For real eigenvalues, their product has magnitude below one, and both roots remain inside the unit interval if neither crosses the ±1 boundary.Source: Appendix A · p. 8
• For complex-conjugate eigenvalues, each squared magnitude equals the determinant, which lies between zero and one under the assumed bounds on τ.Source: Appendix A · p. 8
• Together, the real and complex cases establish that the eigenvalues lie strictly inside the unit circle, so M is Schur stable.Source: Appendix A · p. 8