PaperlayerView exact revision on arXiv

Source-linked AI summary

Gauss--Hermite Quadrature for Gaussian-Mixture Entropy with an Action-Space Hermite Surrogate

Jae Wan Shim

arXiv:2608.21467v2stat.MLcs.ITcs.LG

TL;DR

Gaussian-mixture differential entropy generally lacks a closed-form expression, motivating numerical evaluation. The paper uses componentwise Gauss–Hermite quadrature and develops an action-space Hermite surrogate for repeated continuous-action optimization. On tested benchmarks, low-order quadrature agrees closely with numerical references, while the second-order surrogate outperforms a local Taylor surrogate in the radar benchmark using nine direct objective evaluations per replanning step.

  • Problem

    Gaussian-mixture differential entropy generally has no closed-form expression, requiring approximate evaluation for uncertainty-based objectives in sensor management.

  • Method

    The paper evaluates entropy with direct componentwise Gauss–Hermite quadrature and constructs a Hermite polynomial surrogate in action space for repeated optimization.

  • Results

    Low-order Gauss–Hermite rules agree closely with numerical integration references, and the second-order Hermite surrogate achieves substantially lower surrogate error and optimizer regret than a second-order Taylor surrogate using nine direct evaluations per replanning step.

  • Takeaways & Limitations

    The Hermite surrogate improves detection, inside-gate, and beam-pointing metrics in the tested radar sensor-management benchmark.

  • Takeaways & Limitations

    The surrogate depends on smoothness over the sampled region and may be inaccurate near hard boundaries, discontinuities, association changes, pruning or merging, and mode switching.

Abstract

from arXiv · show

Gaussian distributions are used to model uncertainty in signals and states, and Gaussian mixtures are often used when the underlying distribution is multimodal. Unlike a single Gaussian, a Gaussian mixture generally has no closed-form expression for differential entropy and therefore requires numerical approximation. We propose a Gauss--Hermite quadrature method for evaluating Gaussian mixture differential entropy. The quadrature order controls the numerical resolution of the approximation. The method is evaluated on one- and two-dimensional Gaussian mixture benchmarks against Taylor approximations, analytic entropy bounds, and numerical integration references. For repeated optimization over continuous actions, we also propose a Hermite polynomial surrogate in action space. In a radar pointing benchmark, its second-order form achieves substantially lower surrogate error and optimizer regret than a second-order Taylor surrogate based on local derivatives at the nominal action, while both methods use nine direct objective evaluations per replanning step. The Hermite surrogate also improves pointing performance in the tested benchmark.

1 Introduction

Gaussian mixtures represent multimodal uncertainty in sensing and estimation, but their differential entropy generally requires approximation. The paper evaluates componentwise Gauss–Hermite quadrature and proposes an action-space Hermite surrogate for repeated continuous-action optimization.

  • Motivation: Gaussian mixture models represent non-Gaussian and multimodal distributions arising in tracking, mapping, and Bayesian state estimation.Such models support uncertainty quantification for estimation and decision making.
  • Motivation: Information-driven sensor management requires computationally efficient evaluation and optimization of uncertainty-based objectives represented by Gaussian mixtures.The motivation includes target search, tracking, multi-target management, and nonmyopic sensor control.
  • Prior approaches: Gaussian-mixture differential entropy generally lacks a closed-form expression, motivating Taylor approximations, analytic bounds, numerical integration, Monte Carlo integration, and polynomial approximations.The paper situates its approach among these existing evaluation methods.
  • Contributions: The paper uses direct componentwise Gauss–Hermite quadrature as a deterministic entropy evaluator validated on one- and two-dimensional benchmark families.The validation compares against Taylor approximations, analytic bounds, and numerical integration references.
  • Contributions: For repeated continuous-action optimization, the paper constructs an action-space Hermite polynomial surrogate and compares it with a local second-order Taylor surrogate using the same number of direct evaluations.The radar benchmark uses nine direct objective evaluations per replanning step.

2 Problem Formulation

The paper formulates differential entropy for a Gaussian mixture and identifies the computational obstacle: integrating a logarithm of a weighted sum of Gaussian densities.

  • Entropy objective: The formulation begins with a continuous random vector x ∈ R^d having density g(x), whose differential entropy is evaluated.
  • Gaussian-mixture model: The uncertain state is represented by a Gaussian mixture model with component weights, means, covariances, and Gaussian component densities.Zero-weight components are removed before evaluation, and each covariance is positive definite.
  • Computational difficulty: Direct entropy evaluation is difficult because the integrand contains the logarithm of a weighted sum of Gaussian component densities.This expression generally does not admit analytic integration with respect to the Gaussian-mixture density.

3 Entropy Decomposition and Gaussian Standardization

The entropy integral is decomposed into Gaussian-component expectations, then each component is standardized so its expectation is taken under a standard normal distribution.

  • Entropy decomposition: Substituting the Gaussian-mixture representation into the entropy integral yields an exact decomposition into component-weighted expectations.The finite sum and integral are interchanged in this decomposition.
  • Entropy decomposition: Each component expectation remains analytically intractable because the logarithm retains the full Gaussian mixture.
  • Gaussian standardization: A component covariance factor standardizes the Gaussian variable through Xi = µi + SiY, where Y follows a d-dimensional standard normal distribution.The transformation preserves Xi ∼ N(µi, Ci).
  • Gaussian standardization: After defining standardized component integrands, entropy evaluation reduces to expectations of those functions under the d-dimensional standard normal distribution.The resulting expectations are the quantities targeted by quadrature.

4 Taylor Baselines

The Taylor baselines approximate Gaussian-component entropy expectations locally, optionally splitting broad components before applying second-order expansions. These baselines combine splitting and truncation errors.

  • Baseline construction: The baseline applies Taylor approximations separately to the Gaussian expectations in the entropy decomposition.
  • Local expansion: A second-order Taylor expansion approximates ℓ(Xi) about each component mean µi using the gradient and Hessian of ℓ(x) = log g(x).
  • Local expansion: The zeroth- and first-order entropy approximations coincide because the expected first-order Taylor term vanishes.
  • Local expansion: The resulting second-order approximation retains the expected quadratic contribution around each Gaussian component mean.
  • Component splitting: Component splitting replaces a broad Gaussian with narrower Gaussian subcomponents to improve Taylor accuracy around localized subcomponent means.Flattened weights combine the original component weights with conditional split weights, while the original mixture remains inside the logarithm.
  • Limitations: The split Taylor baseline has two approximation errors: finite-mixture representation error and second-order Taylor truncation error.

5 Hermite Interpretation under the Standard Normal Measure

The paper interprets standardized Gaussian expectations through the orthogonal Hermite basis under the standard normal measure, while evaluating those expectations directly rather than constructing a truncated expansion.

  • Hermite basis: Multivariate probabilists’ Hermite polynomials form a complete orthogonal basis of L2(N(0, Id)).
  • Hermite expansion: The standardized mixture integrand belongs to L2(N(0, Id)), allowing a convergent Hermite expansion.
  • Constant-coefficient interpretation: Each standardized component expectation equals the integrand’s constant Hermite coefficient.
  • Numerical implication: The Hermite interpretation is explanatory: entropy evaluation directly uses Gauss–Hermite quadrature without computing higher-order Hermite coefficients.

6 Gauss–Hermite Entropy Evaluation

The entropy integral is evaluated as standardized Gaussian expectations using a tensor-product Gauss–Hermite rule, with quadrature order controlling numerical resolution and log-sum-exp improving stability.

  • Resolution control: Quadrature order Q is the numerical resolution parameter per dimension, not a Hermite truncation order.
  • Estimator construction: The estimator replaces the Gaussian-mixture entropy integral with L standardized Gaussian expectations evaluated at shared Gauss–Hermite nodes.
  • Numerical stability: Log-sum-exp evaluation avoids direct computation of very small weighted component densities and reduces tail underflow.
  • Computational cost: For dense covariances, the dominant full tensor-product cost scales with outer components, quadrature nodes, mixture components, and O(d^2) Gaussian-density evaluations.
  • Accuracy assessment: Increasing Q or comparing an independent numerical reference provides convergence diagnostics, but not rigorous error bounds.

7 One-Dimensional Parzen-GMM Entropy Benchmark

The benchmark replicates a one-dimensional Parzen-GMM entropy study while distinguishing analytic residual scale diagnostics from finite-sample Parzen-GMM entropy and comparing several evaluation methods against numerical integration.

  • Benchmark setup: The benchmark uses a latent variable X, an independent zero-mean Gaussian measurement noise W, and residual samples reused across candidate coefficients.
  • Residual construction: The replicated residual definition is used throughout the benchmark, with the residual density approximated by an equal-weight Gaussian-kernel Parzen mixture.
  • Residual interpretation: The coefficient 1/a cancels the input-dependent term, but this cancellation does not generally minimize residual entropy because both residual input and noise scales change.
  • Method comparison: The experiment compares entropy methods across a parameter sweep using independent numerical integration as reference, including zeroth- and second-order Taylor and Gauss–Hermite estimates.
  • Interpretive caveat: Analytic Gaussian-residual entropies serve only as scale diagnostics and are not identical to finite-sample Parzen-GMM entropy.
  • Results: GH5 remains nearly coincident with the zero-reference error line, GH3 has small deviations, Taylor-0 has pronounced negative bias, and Taylor-2 has smaller positive bias.

8 Two-Dimensional Gaussian-Mixture Benchmark

The two-dimensional benchmark compares Gauss–Hermite entropy estimates with numerical integration, analytic bounds, and Taylor baselines across mixtures whose overlap and multimodality vary. GH5 stays closest to the numerical reference, while Taylor errors show larger bias or oscillation.

  • Benchmark setup: The benchmark uses a two-dimensional, five-component Gaussian-mixture family whose overlap and multimodality change as the fifth component moves.The continuation parameter varies over a finite grid, and the reference is independently obtained by adaptive numerical integration.
  • Reference calculation: 9.84 × 10−6 nats is the maximum discrepancy between independently integrated reference values and an independent Qref = 41 tensor-product Gauss–Hermite calculation.The independently integrated values are used as the reference curve.
  • Compared methods: The comparison includes the Jensen lower bound, refined and single-Gaussian upper bounds, unsplit and split second-order Taylor approximations, and GH3 and GH5.The refined upper bound never exceeds the single-Gaussian bound by construction.
  • Results: GH5 exhibits substantially smaller signed-error amplitude than GH3 and remains close to the numerical reference over the parameter sweep.The error plot omits analytic bounds because they are intended to bracket entropy rather than approximate it symmetrically.
  • Results: The unsplit second-order Taylor approximation has pronounced oscillatory errors, whereas the split version shows a predominantly negative bias.The Gauss–Hermite estimator requires neither derivatives of log gc(x) nor component splitting.

9 Hermite Surrogate for Repeated Action Evaluation

The paper constructs a Hermite polynomial surrogate in standardized action coordinates for repeated optimization of an entropy-based sensing objective. The surrogate uses spatially distributed objective evaluations for projection, but its accuracy depends on local smoothness and computational cost grows with action dimension.

  • Surrogate purpose: The action-space Hermite surrogate is a separate approximation layer from the state-space Gauss–Hermite entropy evaluator.It targets repeated evaluation and optimization of an entropy-based objective over continuous sensing actions.
  • Objective: The expected posterior-entropy objective averages posterior entropy over the predictive measurement density before selecting an entropy-minimizing sensing action.After an actual measurement is observed, no outer expectation over the measurement is required.
  • Action coordinates: The coordinate map separates physical action units and admissibility constraints from dimensionless coordinates used in the Hermite expansion.Injectivity gives each mapped action a unique coordinate representation, while the global map ensures projected coordinates correspond to admissible actions.
  • Hermite projection: The nonintrusive projection retains Hermite terms through total order R and estimates coefficients from objective values at Gauss–Hermite nodes.Unlike a local Taylor quadratic, the exact Hermite projection uses Gaussian-weighted information distributed over standardized action space.
  • Applicability and limitations: The surrogate must be reconstructed when the belief or model changes appreciably, and its accuracy depends on smoothness over sampled and optimized action regions.Low-order surrogates may be inaccurate with hard boundaries, discontinuous gating, mode switching, or other nonsmooth operations.
  • Applicability and limitations: The tensor-product quadrature cost Mu = Qdu grows exponentially with action dimension, so sparse-grid or adaptive rules may be needed when du is not small.Increasing polynomial order is useful only when the quadrature rule and direct evaluations resolve the associated higher-order variation.

10 Closed-Loop Radar Pointing Benchmark for the Action Surrogate

The closed-loop radar benchmark evaluates a quadratic Hermite action-space surrogate against a derivative-calibrated second-order Taylor surrogate under matched sensing conditions and nine direct objective evaluations. The Hermite surrogate achieves lower approximation error and regret, with improvements carrying through to pointing and detection performance.

  • Benchmark setup: The benchmark isolates beam-pointing decisions with fixed dwell time, while all action-selection methods use the same Gauss–Hermite state-space entropy evaluator.The comparison targets the action-space surrogate rather than alternative entropy estimators.
  • Benchmark setup: The radar tracker uses three maneuver hypotheses—left turn, straight motion, and right turn—with initial weights 0.25, 0.50, and 0.25.The fixed turn-rate hypotheses are +6°/s, 0, and −6°/s, while truth maneuvers can exceed these rates.
  • Surrogate construction: The quadratic Hermite surrogate retains six total-degree-two modes from nine tensor-product evaluations as a Gaussian-weighted least-squares projection.It need not interpolate all nine objective values; the construction is a weighted projection onto the total-degree-two polynomial space.
  • Surrogate accuracy: The mean surrogate RMSE decreases from 2.7190 to 0.5610, and the paired NRMSE reduction is 0.6824 with 95% bootstrap confidence interval [0.6610, 0.7043].All 50 trial-cluster differences favor the Hermite surrogate for RMSE, NRMSE, optimizer regret, and standardized action error.
  • Surrogate accuracy: The mean Spearman rank correlation with the directly evaluated objective increases from 0.6554 to 0.9464.Only 1.44% and 1.15% of diagnostic epochs are excluded from NRMSE aggregation for the two methods.
  • Closed-loop performance: The overall detection rate increases from 0.8850 for Taylor to 0.9429 for Hermite, a paired increase of 0.0579 with 95% bootstrap confidence interval [0.0510, 0.0650].The detection-rate advantage grows from approximately 0.037 at 2°/s to 0.091 at 12°/s; temporary-loss reductions do not support an overall loss-prevention claim because the interval includes zero.
  • Computational cost: Both surrogates use nine direct objective evaluations per replanning step, whereas the 81-point direct search requires about 359 ms versus approximately 39.4 ms for either surrogate.The transferable computational comparison is the reduction from 81 direct evaluations to 9, since absolute timings depend on implementation and hardware.

11 Discussion

The paper interprets Gauss–Hermite entropy evaluation as direct quadrature over standardized Gaussian expectations, while using higher-order Hermite terms for action-space surrogates. Results support accurate low-order entropy estimates and improved radar surrogate performance, with dimensionality limiting full tensor-product rules.

  • Entropy evaluation: Low-order Gauss–Hermite rules closely track numerical references on tested one- and two-dimensional Gaussian mixtures.Accuracy depends on standardized mixture geometry, including component separation, covariance anisotropy, and overlap.
  • Entropy evaluation: Quadrature order controls numerical resolution, and successive-order differences provide a practical diagnostic for a given mixture family.The diagnostic is based on differences such as |bHQ − bHQ+2|.
  • Hermite interpretation: The entropy evaluator uses direct quadrature rather than a truncated Hermite expansion, whereas the action surrogate retains higher-order Hermite terms.The action-space construction is interpreted as a nonintrusive Hermite polynomial-chaos or pseudospectral surrogate.
  • Radar benchmark: The radar benchmark reports lower surrogate error, optimizer regret, standardized action error, and normalized beam error, alongside higher detection and inside-gate rates, for Hermite than local Taylor surrogates.Temporary and terminal loss outcomes are less statistically resolved, and position-RMSE effects are secondary and less uniform across trials.
  • Limitations and extensions: Full tensor-product Gauss–Hermite rules face exponential growth with dimension, while difficult or anisotropic mixtures may require additional nodes.Sparse grids, dimension-adaptive rules, and problem-specific transformations are identified as natural extensions.
  • Extensions: The Gaussian-expectation decomposition also supports cross-entropy and Kullback–Leibler divergence calculations when the second density is evaluable at quadrature nodes.These extensions retain the distinction between exact decomposition and numerical quadrature approximation.

12 Conclusion

The paper evaluates Gaussian-mixture entropy with direct componentwise Gauss–Hermite quadrature and constructs an action-space Hermite surrogate for repeated continuous-action optimization. The evaluator agrees closely with numerical references on tested low-dimensional mixtures, while the radar surrogate outperforms a local Taylor baseline at equal evaluation count, within the study’s scope.

  • 12 Conclusion: Direct componentwise Gauss–Hermite quadrature evaluates standardized Gaussian expectations without derivative evaluation or component splitting.On the tested one- and two-dimensional benchmark families, low-order rules agree closely with numerical integration references.
  • 12 Conclusion: The second-order Hermite action surrogate achieves substantially lower surrogate error and optimizer regret than the second-order local Taylor surrogate using nine direct evaluations per replanning step.It also improves detection, inside-gate, and beam-pointing metrics in the tested radar benchmark.
  • 12 Conclusion: The entropy accuracy results are limited to the tested one- and two-dimensional benchmark families and do not establish a general convergence rate with quadrature order.Future work includes sparse-grid and adaptive quadrature, systematic convergence studies, and higher-dimensional action surrogates.
Loading 2608.21467v2…