Source-linked AI summary
A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence
Yi Li, Eric Perlman, Minping Wan, Yunke Yang, Charles Meneveau, Randal Burns, Shiyi Chen, Alexander Szalay, Gregory Eyink
TL;DR
The paper addresses how turbulence researchers can analyze massive DNS archives without downloading the full datasets or relying on colocated HPC resources. It builds a remotely accessible database with in-database processing, validates its analyses, and applies it to Lagrangian intermittency dynamics. The pressure and viscous contributions differ substantially, supporting separate closure-modeling strategies.
Problem
Existing turbulence data-sharing approaches require downloading most data or accessing substantial HPC resources, while unclosed terms prevent quantitative intermittency modeling.
Method
The authors construct a 27 Terabyte database of 1024^4 forced-isotropic-turbulence DNS data with Web-service access and in-database analysis functions, then compare database calculations with in-core analyses.
Results
The pressure and viscous terms in the advected delta-vee system show markedly different behaviors and oppose excessive intermittency growth, requiring separate modeling strategies.
Takeaways & Limitations
The database provides a public platform for remote turbulence research and supports physically differentiated closure modeling for Lagrangian velocity-increment intermittency.
Takeaways & Limitations
The analysis is restricted to unfiltered data because filtering operations are not yet supported, and the forcing term is unavailable directly from the database.
Abstract
from arXiv · showhide
A public database system archiving a direct numerical simulation (DNS) data set of isotropic, forced turbulence is described in this paper. The data set consists of the DNS output on $1024^3$ spatial points and 1024 time-samples spanning about one large-scale turn-over timescale. This complete $1024^4$ space-time history of turbulence is accessible to users remotely through an interface that is based on the Web-services model. Users may write and execute analysis programs on their host computers, while the programs make subroutine-like calls that request desired parts of the data over the network. The users are thus able to perform numerical experiments by accessing the 27 Terabytes of DNS data using regular platforms such as laptops. The architecture of the database is explained, as are some of the locally defined functions, such as differentiation and interpolation. Test calculations are performed to illustrate the usage of the system and to verify the accuracy of the methods. The database is then used to analyze a dynamical model for small-scale intermittency in turbulence. Specifically, the dynamical effects of pressure and viscous terms on the Lagrangian evolution of velocity increments are evaluated using conditional averages calculated from the DNS data in the database. It is shown that these effects differ considerably among themselves and thus require different modeling strategies in Lagrangian models of velocity increments and intermittency.
1 Introduction
The paper addresses the difficulty of turning massive turbulence simulations into usable knowledge and develops a public database-based approach for remote analysis. It applies the database to modeling unclosed pressure and viscous effects in Lagrangian velocity-increment dynamics.
- Motivation: Petascale turbulence simulations can generate datasets of about 160 petabytes, creating major challenges for extracting and transmitting useful knowledge.The paper motivates database technology as a way to identify patterns, build reduced-order descriptions, and reduce data movement.
- Motivation: Existing DNS data-sharing efforts generally require downloading most data or obtaining substantial HPC resources where the data are stored.The paper argues that these approaches do not fully exploit database technologies for flexible analysis.
- Contribution: The authors construct a 27 Terabyte database containing a 1024^4 space-time history of forced isotropic turbulence accessible through an Internet user interface.The archive stores velocity and pressure fields from 1024^3 grid points across 1024 time samples and supports in situ differentiation and interpolation.
- Scientific background: Small-scale intermittency produces strong fluctuations, non-Gaussian PDF tails, and velocity-increment scaling exponents that depart from classical Kolmogorov predictions.These departures make quantitative prediction of intermittency a major turbulence-research challenge.
- Scientific background: The truncated advected delta-vee system reproduces several intermittency trends but diverges because important pressure and viscous terms are omitted.The paper therefore analyzes these unclosed terms using DNS data and their probability fluxes.
- Paper scope: The paper organizes the database description, dataset documentation, validation calculations, and application to Lagrangian intermittency modeling into successive sections.The validation includes energy spectra, velocity-gradient PDFs, and the joint PDF of the Q and R invariants.
2 Architecture of the database cluster
The database cluster stores a large forced-isotropic-turbulence DNS archive and exposes it through remote services, in-database processing, and locality-aware data organization. Its design combines public access, flexible computation, platform compatibility, and efficient parallel execution.
- Design requirements: The cluster is designed to provide public network access, in-database processing, platform compatibility, reuse of existing tools, and efficiency.These requirements define the user-centered goals of the database architecture.
- Cluster architecture: The archive stores 1024^3 DNS data over 1024 time steps, totaling 27 terabytes, with data partitioned spatially across database nodes.The cluster uses heterogeneous multi-core servers running 64-bit Microsoft Windows 2003 Server.
- Cluster architecture: A Web-service data access server mediates user requests, distributes subrequests across database nodes, and enables parallel computation near the stored data.The architecture follows a “move the program to the data” principle to reduce data movement and improve efficiency.
- Turbulence dataset: The DNS uses a pseudo-spectral method on a 1024^3 periodic grid, with forcing at wavenumbers up to 2 and a dealiased maximum wavenumber of about 482.The stored physical-space archive contains velocity and pressure fields sampled every 0.002 time units for 2.048 time units.
- Processing function set: The database provides stored processing procedures through SQL Server CLR integration, balancing generality with efficiency for client-specific turbulence analyses.The processing functions are implemented in C# and expose compact tools for operations such as differentiation and interpolation.
- Validation and dataset checks: The database’s energy spectra, kinetic-energy time series, and Taylor-scale Reynolds-number series are evaluated over the stored interval to characterize the archived simulation.The spectrum is averaged from t = 0 to 2.048, while database data are shown as solid-line portions in the time-series figures.
- Data indexing, partition, and workload schedule: Z-order indexing maps three-dimensional grid points to a one-dimensional layout that preserves locality for coherent-region access and filtering.The same ordering supports spatial partitioning, atom-based I/O, caching, and workload scheduling.
3 Preliminary numerical tests
The database tests compare remotely queried turbulence statistics with in-core calculations, including spectra, gradient PDFs, and invariant PDFs. The results show close agreement while also characterizing interpolation effects and access-time scaling.
- Test design: The tests compare database-query calculations of energy spectra, velocity-gradient PDFs, and the joint PDF of velocity-gradient invariants with traditional in-core analysis.Database queries use finite differences and interpolation, while in-core calculations use Fourier-space spectral derivatives.
- Energy spectra: The database spectra agree with in-core spectra, confirming the correctness of the loaded velocity data at grid points.The comparison uses 512 grid lines with 1024 points per line and no interpolation error at grid locations.
- Energy spectra: The spectra exhibit a narrow inertial range where the longitudinal and rescaled transverse spectra overlap, with deviations toward the viscous range.The spectra extend to about wavenumber 482 because of dealiasing.
- Interpolation: Sixth-order Lagrange interpolation extends the plotted spectral range to about 2000 but produces oscillating high-wavenumber lobes containing little energy.The oscillations occur between the maximal resolved wavenumber, about 482, and 2000.
- Access performance: The energy-spectrum calculations take about 16 minutes for 512 × 1024 points and 26 minutes for 512 × 4096 points.These timings are reported for the initial implementation, which the authors note still allows further optimization.
- Access performance: For remote requests, access time depends on point arrangement, interpolation, and request size, scaling approximately as N_p^0.9 for cubic arrangements and N_p^0.4 for lines or random points.Cubic arrangements are fastest for moderate-size requests because data are concentrated in fewer stored atoms.
4 Analysis of the advected delta-vee system
The paper uses the turbulence database to formulate and analyze probability fluxes in the advected delta-vee system. The fluxes reveal distinct roles for self-interaction, pressure, viscous, and Q− terms in velocity-increment intermittency.
- System formulation: The advected delta-vee system describes Lagrangian evolution of longitudinal and transverse velocity increments, with nonlinear self-interaction terms closed and Q−, Y, and Z unclosed.The increments are defined along evolving line elements over a fixed distance ℓ.
- System formulation: The joint-PDF equation is written as ∇·W + S = 0, decomposing probability flux into self-interaction, Q−, pressure, viscous, and forcing contributions.For stationary turbulence, the unsteady term vanishes.
- Database analysis: The database analysis computes the joint PDF and conditional flux averages from queried DNS data, using 80 × 80 phase-space bins and about half a million random datapoints.The forcing flux is omitted, while the viscous Laplacian is evaluated locally with a fourth-order central finite-difference approximation.
- Self-interaction effects: The closed self-interaction flux produces negative skewness in δu through self-amplification and amplifies transverse fluctuations through cross amplification.The self-interaction terms alone cannot maintain a stationary joint distribution.
- Unclosed-term effects: The Q− term generally opposes self-interaction, whereas the pressure Hessian reverses its flux direction and reduces intermittency; these effects can also strengthen intermittency in selected regions.The Q− flux is parallel to the δu axis because its δv component is zero.
5 Conclusions
The paper presents a turbulence database system for remote access to a 27 Terabyte DNS space-time history and demonstrates its use in analyzing velocity-increment intermittency. The analysis finds that unclosed terms counter excessive intermittency growth and have markedly different effects.
- 27 Terabytes of forced-isotropic-turbulence DNS data are encapsulated in a complete 1024^4 space-time history accessible remotely.The system applies database technologies to provide remote access to the data set.
- The database architecture covers the user interface, processing functions, data indexing and partitioning, and workload scheduling.Test cases demonstrate system usage and check the methods against stated verification goals.
- The advected delta-vee system is analyzed by accessing the database from a remote desktop computer.
- Unclosed terms oppose the excessive growth of intermittency in velocity increments produced by nonlinear self-interaction terms.
- The isotropic pressure Hessian, deviatoric pressure Hessian, and viscous terms show markedly different effects.
A Equation for the joint PDF of the longitudinal and transverse velocity increments
The appendix derives the joint PDF equation for longitudinal and transverse velocity increments from their dynamics and the evolving line-element length. It introduces a measure correction to remove trajectory-weighting bias and states the boundary assumptions used in the derivation.
- The joint PDF of δu and δv is derived from their dynamic equations and the equation for the evolving line-element length r(t).
- Velocity increments are defined along evolving line elements that concentrate near flow-stretching directions, biasing equal-trajectory statistics.
- A measure-correction procedure is introduced to obtain unbiased statistics for velocity increments in the (δu, δv) phase space.
- Figures 16–18 plot probability-flux vectors generated by the Q−term, anisotropic pressure Hessian, and viscous diffusion term, respectively.The Q−term vectors are parallel to the δu axis, with contours showing their magnitudes.
- The derivation relates the corrected PDF P to the traditional three-variable PDF P3 through normalization and integration over r.
- The derivation assumes vanishing boundary fluxes and that δuP3 approaches zero faster than r2 as r tends to zero.
- After substituting the increment dynamics and rearranging, the final joint-PDF equation includes a measure-correction term and assumes ℓ = 1.
B Documentation: database spatial differentiation
The database documentation describes spatial-differentiation options for velocity components or pressure, using grid widths in the x and y directions. It establishes the notation used by the differentiation functions.
- The spatial-differentiation functions operate on any velocity component u, v, or w, or on pressure p.
- The differentiation documentation denotes the grid widths in the x and y directions by ∆x and ∆y.
- The appendix provides details about the various spatial-differentiation options implemented in the database cluster.
B.1 Options for GetVelocityGradient and GetPressureGradient
The database provides derivative evaluations using finite-difference options, with spatial data arranged along coordinate directions and edge replication supporting high-order methods.
- The spatial data points are illustrated along the x direction, with the same approach used in y and z.
- 4th-order Lagrange Polynomial interpolation uses four replicated data points on each side.
- Four replicated data points on each side enable the highest-order finite-difference option available.
B.2 Options for GetVelocityLaplacian, GetVelocityHessian and GetPressureHessian
The database documents second-derivative evaluations for single and mixed spatial directions, including centered finite-difference options of different orders.
- Second derivatives are evaluated along single directions and through mixed derivatives illustrated on the x-y plane.The same approach extends to y and z directions and to the x-z and y-z planes.
- FD6 denotes 6th-order centered finite differencing.
- FD8 denotes 8th-order centered finite differencing.
C Documentation: Database Spatial Interpolation Options
The database implements several spatial interpolation and derivative-retrieval options for velocity- and pressure-related quantities across the three coordinate directions.
- Spatial interpolation options are documented for the database cluster.
- Nearest-neighbor retrieval returns the value at the data point closest to each coordinate, rounding independently by direction.
- 4th-, 6th-, and 8th-order Lagrange Polynomial interpolation are available along each spatial direction.The interpolation direction θ may be x, y, or z.
- Derivative options retrieve 4th-, 6th-, or 8th-order finite-difference values at the nearest data point.
- 4th-order finite-difference values at data points can be spatially interpolated with 4th-order Lagrange Polynomials.The same coordinate-direction notation applies, with θ equal to x, y, or z.
D Documentation: Database Temporal Interpolation Options
Temporal access supports nearest-point retrieval and interpolation between stored time samples, including a four-point cubic Hermite scheme.
- Temporal functions operate on velocity, pressure, their derivatives, Hessians, Laplacians, and a stored time increment ∆t.
- Nearest-time retrieval returns the value at the data point closest to the requested time, rounding up or down.
- The nearest stored time index is selected using n = int(t′/∆t + 1/2).
- Cubic Hermite interpolation uses the two nearest time points and centered finite-difference endpoint derivatives, totaling four temporal points.
- The temporal interpolation is represented by a cubic polynomial f(t′) between consecutive times t_n and t_n+1.