Nonlinear filtering stabilizations for the quasi-geostrophic equations

Lander Besabe; Sachin Kumar; Annalisa Quaini

Source-linked AI summary

Nonlinear filtering stabilizations for the quasi-geostrophic equations

Lander Besabe, Sachin Kumar, Annalisa Quaini

arXiv:2609.16983v1Sep 15, 2026math.NAphysics.comp-ph

TL;DR

The paper addresses the high cost of resolving the Munk scale in QGE simulations by developing filtering-based LES stabilization. It proposes and interprets EFR alongside nonlinear BV-Bardina regularization, finding improved coarse-mesh accuracy and stability, with deconvolution-based EFR offering the best reported balance and about 64-fold lower cost than DNS in the Mediterranean test.

Problem
Resolving the Munk scale requires fine meshes and high computational cost, motivating LES models that represent unresolved scales on coarser meshes.
Method
The paper develops a three-step QGE EFR algorithm with nonlinear Helmholtz filtering and indicator functions, and compares it with nonlinear BV-Bardina regularization.
Results
EFR and nonlinear BV-Bardina significantly improve coarse-mesh accuracy and stability; deconvolution-based EFR gives the best balance and costs about 64 times less than DNS in the Mediterranean test.
Takeaways & Limitations
The deconvolution-based QGE-EFR model is the strongest reported option for balancing accuracy, stability, and computational efficiency across the tested cases.
Takeaways & Limitations
The loosely coupled nonlinear BV-Bardina algorithm may be unstable for highly convection-dominated flows and some discretization parameters, requiring fixed-point subiterations that add computational time.

Abstract

from arXiv · show

Numerical simulations of ocean flows typically require fine computational meshes to resolve the Munk scale, leading to high computational costs. Filtering-based large eddy simulation (LES) provides a way to relax the mesh size requirement by modeling the effects of the unresolved scales. For the implementation of this strategy, we propose a three-step algorithm called Evolve-Filter-Relax (EFR) that requires (i) the solution of a QGE problem, (ii) a nonlinear Helmholtz filter for the potential vorticity field leveraging an indicator function, and (iii) a final relaxation step. We show that the EFR algorithm can be interpreted as a splitting scheme for a perturbed QGE problem with additional dissipation and provide a practical choice for the relaxation parameter. For comparison, we also investigate a nonlinear Bardina regularization of the QGE. Numerical results on a classical benchmark show that both the EFR approach and the nonlinear Bardina regularization significantly improve the accuracy and stability of coarse mesh simulations with no LES model. Additionally, the EFR method with a deconvolution-based indicator function delivered the best balance between accuracy, stability, and computational efficiency in a test case involving a more realistic geometry (Mediterranean Sea).

1. Introduction

Fine meshes are needed for challenging multiscale QGE ocean simulations, motivating LES strategies such as EFR and nonlinear filtering to model unresolved scales.

1. Introduction: Small Munk scales and broad flow structures make QGE simulations computationally expensive because meshes must resolve the smallest scales.QGE retain the computational challenges of ocean flows even though they are simplified equations.
1. Introduction: LES reduces computational cost by using coarse meshes together with subgrid-scale models for unresolved scales.A common option is a substantially enhanced eddy viscosity, while more sophisticated models adapt viscosity to flow features and mesh.
1. Introduction: Prior QGE LES work includes eddy parametrization, alpha and Bardina regularizations, approximate deconvolution, coarse-grid projection, closure models, and differential filtering.These approaches target accuracy and stability for under-resolved QGE simulations.
1. Introduction: EFR applies evolve, filter, and relax steps to coarse-mesh QGE simulations, while its application to QGE is presented as new.The governing equations are first evolved, the result is filtered, and filtered and unfiltered solutions are combined.
1. Introduction: The paper derives an additional-dissipation interpretation and practical relaxation choice for EFR, then compares it with linear and nonlinear filter variants and BV-Bardina models.Accuracy and stability are assessed using flow fields, kinetic energy evolution, and kinetic energy spectra.

2. Problem definition

The paper defines a nondimensional QGE model for rotating ocean flow and explains why resolving the Munk scale remains costly, motivating coarse-mesh LES with SGS modeling.

2. Problem definition: The QGE model describes a rotating fluid layer under near hydrostatic and geostrophic balance with a linearized beta-plane Coriolis frequency.The model uses constant depth, temperature, and density, with Coriolis variation represented by the beta-plane approximation.
2. Problem definition: The nondimensionalization introduces a Reynolds number based on eddy viscosity, although calling it Reynolds number is technically an abuse of terminology.The actual Reynolds number uses the fluid’s molecular viscosity.
2. Problem definition: The nondimensional potential vorticity is q = Ro ω + y, where ω is nondimensional vorticity and Ro is the Rossby number.The nondimensional formulation also introduces Reynolds and Rossby numbers.
2. Problem definition: The QGE problem requires appropriate boundary conditions and initial data to close the governing equations.The paper states that these conditions are imposed following prior QGE studies.
2. Problem definition: Resolving the Munk scale requires high computational cost, so LES uses h > δM together with an SGS model for unresolved-scale effects.The coarse-mesh formulation results from spatial filtering.

3. BV-α and BV-Bardina models

This section develops BV-α and BV-Bardina regularizations, including nonlinear filtering variants, and identifies stability and computational trade-offs in their algorithms.

3. BV-α and BV-Bardina models: BV-Bardina replaces transported vorticity with filtered vorticity, regularizing nonlinear advection and potentially capturing backscattering of energy.The model introduces stronger regularization than BV-α and has previously provided good coarse-mesh QGE approximations.
3. BV-α and BV-Bardina models: Nonlinear filtering improves accuracy on very coarse meshes compared with the mathematically and computationally convenient linear filter.The nonlinear filter uses an indicator function to adapt regularization to the flow.
3. BV-α and BV-Bardina models: The indicator function ranges from near zero where regularization is unnecessary to near one where regularization is needed.It may depend on potential vorticity, stream function, or both.
3. BV-α and BV-Bardina models: The nonlinear BV-Bardina model is discretized with a segregated algorithm that sequentially computes filtered and unfiltered vorticity quantities.The displayed implementation includes separate steps for finding filtered vorticity and vorticity at each time level.
3. BV-α and BV-Bardina models: Fixed-point subiterations can improve stability for highly convection-dominated flows, but the improved stability increases computational time.The tighter coupling between stream function and vorticity contributes to possible instability in the loosely coupled algorithm.

4. The Evolve-Filter-Relax algorithm for the QGE

QGE-EFR adds regularization through sequential evolve, filter, and relax steps, with filtering controlled by an indicator function and relaxation parameter. The method is interpreted as a splitting scheme for a perturbed QGE with additional dissipation, and its perturbations vanish under refinement.

Algorithm: QGE-EFR sequentially evolves the unregularized QGE, filters the potential-vorticity field, and relaxes the result to mitigate over-smoothing.The filtering step uses a nonlinear Helmholtz operator, while the evolve step can be decoupled by making the convective velocity explicit.
Interpretation: The EFR algorithm is a splitting scheme for a perturbed QGE containing an additional dissipative term characteristic of LES.The perturbed formulation motivates referring to the algorithm as the QGE-EFR model.
Consistency: All EFR perturbations are multiplied by χ and vanish as ∆t and h tend to zero, recovering the original QGE problem.The dominant χ-weighted perturbation is the first of three terms; the other two vanish faster as h tends to zero.
Indicator functions: The indicator function determines where and how strongly filtering acts, ranging from uniform diffusion to selective filtering in regions requiring greater stabilization.The Leith-type indicator targets high potential-vorticity gradients, the Smagorinsky-type indicator targets large velocity gradients, and the deconvolution-based indicator is included for realistic flows.
Parameter choice: The relaxation parameter χ is chosen by matching coarse-mesh effective dissipation to the viscous contribution on a mesh resolving the Munk scale.A simplified time-independent formula uses Re, δM, h, and ∆t, but is effective only when the indicator magnitude is close to one.

5. Numerical Results

Across two double-gyre experiments, nonlinear BV-Bardina and QGE-EFR filtering improved coarse-mesh QGE solutions, while their best indicator and filter settings depended on resolution and Reynolds number.

5.1. The linear and nonlinear BV-Bardina models: BV-Bardina models recovered the four-gyre pattern and realistic magnitude ranges, but remained less accurate for potential vorticity in narrow boundary regions.At h = 1/16, the unfiltered QGE missed the four-gyre pattern and had stream-function peaks about 70 times larger than DNS; refinement to h = 1/32 substantially improved reconstruction.
5.1. The linear and nonlinear BV-Bardina models: BV-Bardina filtering corrected the coarse QGE’s excessive kinetic energy and produced energy spectra closer to DNS, with correct resolved-scale slopes and smaller discrepancies after mesh refinement.Indicator-function differences were smaller than the discrepancy between filtered and unfiltered coarse QGE, suggesting that filtering itself dominated the recovery of the energy distribution.
5.1. The linear and nonlinear BV-Bardina models: Indicator choice created a practical trade-off: aS was most robust across mesh and physical-parameter changes, whereas aLeith gave closer spectra at increased computational cost and alin supplied extra dissipation in difficult cases.In experiment 2, aLeith best matched the DNS spectrum among BV-Bardina variants, while aS was identified as the most reliable general choice and alin as a more stable alternative for complex cases.
5.2. The QGE-EFR model: QGE-EFR models recovered the four-gyre structure, avoided boundary oscillations in potential vorticity, and produced stable kinetic-energy behavior on coarse meshes.For the more challenging experiment 2, sufficient dissipation required α = 2h; the deconvolution-based indicator generally performed best, while the linear indicator was more dissipative.

6. Conclusion

The paper develops and evaluates EFR and nonlinear BV-Bardina regularizations for under-resolved QGE simulations. Both improve coarse-mesh accuracy and stability, while deconvolution-based EFR offers the strongest overall balance in the Mediterranean test.

6. Conclusion: QGE-EFR is implemented as evolve, filter, and relax steps and interpreted as a splitting scheme for a perturbed QGE with additional dissipation.The nonlinear filter uses an indicator function to tune the amount and location of effective eddy viscosity.
6. Conclusion: The nonlinear BV-Bardina comparison uses indicator functions to regulate filtering and provides an alternative regularization strategy for QGE.The study assesses its performance alongside QGE-EFR using multiple indicator choices.
6. Conclusion: Both QGE-EFR and nonlinear BV-Bardina significantly improve coarse-mesh accuracy and stability over QGE without a subgrid-scale model.On the double-gyre benchmark, both recover time-averaged flow structures, realistic kinetic energy levels, and accurate energy spectra.
6. Conclusion: The deconvolution-based indicator delivers the best balance between accuracy, stability, and computational efficiency among the investigated models.This choice also performs well in the realistic Mediterranean Sea test case.
6. Conclusion: Mediterranean simulations with deconvolution-based QGE-EFR capture the large-scale gyres and kinetic-energy evolution at about 64 times lower cost than DNS.The comparison uses the realistic Mediterranean geometry and regularized coarse-mesh simulations.