PaperlayerView exact revision on arXiv

Source-linked AI summary

What Bloats Your Floats? Right-Sizing Numerical Precision for Scientific Computing

Pratyai Mazumder, Alexandru Calotoiu, Marcin Copik, Yakup Koray Budanaz, Shriram Chandran, Torsten Hoefler

arXiv:2610.09089v1cs.DC

TL;DR

PDE simulations may not benefit from FP64 bits masked by discretization error, yet validating lower precision at scale remains challenging. The paper proposes a refinement-based SNR criterion and automates dataflow-level precision reduction; its weather and climate evaluations report GPU speedups up to 1.8× while preserving macroscopic fidelity.

  • Problem

    Precision tuning lacks a scalable validation criterion for PDE simulations, and same-grid high-precision references can measure arithmetic differences rather than physical accuracy.

    Source: §I · p. 1, D. Moving Beyond Arithmetic Baselines · p. 4

  • Method

    The framework uses a discretization-floor-based SNR criterion and uniformly lowers dataflow subgraphs to a target precision, validating their output against the SNR baseline.

    Source: §III · p. 6, §VII · p. 12

  • Results

    Up to 1.8× GPU speedup is reported for FP32 on ICON velocity tendencies and CLOUDSC while preserving macroscopic numerical fidelity.

    Source: §VII · p. 12

  • Takeaways & Limitations

    The safe precision boundary depends on application and resolution, making an automated criterion useful for precision reduction.

    Source: §VII · p. 12

    The criterion assumes discretization-limited solvers with refinable runs and meaningful output variance; it does not cover computations without a discretization to refine against.

    Source: §VI · p. 12

Abstract

from arXiv · show

While FP64 (binary64) remains the standard representation of real numbers in scientific computing, hardware trends increasingly favor low-precision formats optimized for AI workloads. This shift prioritizes throughput and energy efficiency over full-precision capabilities. We challenge the necessity of high-precision arithmetic in PDE-governed simulations, where spatial and temporal discretization errors introduce a physical noise floor that masks the lower-order bits of the FP64 format, rendering them irrelevant. We propose a criterion based on the Signal-to-Noise Ratio (SNR) with respect to spatial and temporal refinements to evaluate the impact of safe precision reduction on scientific computations. Finally, we present an automated dataflow-centric workflow for targeted lowering within mixed-precision subgraphs in complex scientific applications. Large-scale evaluations of weather and climate applications show that strategic precision reduction on GPUs achieves up to 1.8x speedup without compromising physical fidelity.

I. INTRODUCTION

The paper argues that PDE simulations often carry more precision than their discretizations can use, but safe reduction requires scalable validation. It proposes an SNR criterion and an automated dataflow workflow, then relates the practical precision boundary to grid resolution.

I. INTRODUCTION

  • FP64 remains costly on bandwidth-bound PDE solvers, while discretization error can mask its lower-order bits and production codes lack scalable tools to validate downcasting.Existing reference-based metrics can be misleading because both test and reference share discretization error; pointwise, relative, and distributional alternatives also have limitations for automated tuning.Source: §I · p. 1

C1. A New Perspective on Precision Tuning in Scientific Computing:

  • The proposed variance-based SNR uses the discretization noise floor, estimated through spatial and temporal refinements, instead of a high-precision reference.The criterion is described as scale-free, tolerant of nonlinear dynamics, and requiring no reference beyond FP64.Source: C1. A New Perspective on Precision… · p. 2

C2. Automated Dataflow-Level Precision Tuning in Production-Scale Weather and Climate Models:

  • Up to 1.8× GPU speedup is achieved on weather and climate kernels while preserving macroscopic numerical fidelity.The framework validates precision reductions directly on dataflow graphs against the proposed threshold.Source: C2. Automated Dataflow-Level Precision… · p. 2

II. DISCRETIZATION NOISE AND PRECISION LIMITS

  • At practical grid resolutions, FP64-to-FP32 error penalties can be nearly negligible, while extreme refinement can make the penalty catastrophic.For the Burgers’ equation example, the penalty is almost negligible at h^-1 ≤ 500 and catastrophic at h^-1 = 10 000.Source: §II · p. 2

A. The Marginal Utility of Precision

The marginal benefit of adding precision beyond the point where round-off matches discretization error declines exponentially. The section connects that result to the behavior of total error and the practical limits of precision tuning.

A. The Marginal Utility of Precision

  • Under the PQN framework, round-off error is modeled as zero-mean noise uncorrelated with discretization error, with variance proportional to ε².This lets the second moment of total error be expressed through the discretization error and round-off variance.Source: A. The Marginal Utility of Precision · p. 2
  • Pointwise metrics can fail at shocks and saturate under chaotic trajectory divergence, limiting their usefulness for assessing reduced precision.L1 offers partial tolerance by integrating |e| rather than e².Source: A. The Marginal Utility of Precision · p. 3
  • The marginal variance reduction from the k-th additional bit after ε* decays as O(4^-k).The total error converges exponentially toward the discretization floor, so additional significand bits provide diminishing returns for macroscopic fidelity.Source: A. The Marginal Utility of Precision · p. 3

B. Physical Limits and Error Quantification

The section defines a variance-based SNR to assess numerical fidelity while discounting static bias. Its predicted and observed behavior is bounded by spatial and temporal discretization floors, with round-off degrading SNR beyond those limits.

B. Physical Limits and Error Quantification

  • For the Burgers’ equation example, spatial refinement follows the discretization limit at practical resolutions; SNR collapses only under extreme over-refinement when round-off overtakes that limit.Temporal refinement improves SNR until the spatial floor caps it; the figure also shows absorption plateaus before full degradation.Source: Figure 2 · p. 3
  • Variance-based SNR discards static bias to focus on the anomalies and fluctuations that govern simulated dynamics.The conventional power-based SNR and L2 norm are sensitive to harmless static biases.Source: B. Physical Limits and Error… · p. 4
  • SNR varies with grid refinement as a balance between spatial discretization variance and round-off variance, with a peak whose location depends on precision.The scaling uses problem-dependent constants that may be analytically unknowable.Source: B. Physical Limits and Error… · p. 4

C. An Example: SNRs in 1D Burgers’ Equation

In the 1D Burgers’ FTCS example, practical-resolution FP32 and FP64 solutions reach the same analytical SNR floor, although same-grid numerical references suggest a large gap. The apparent discrepancy reflects arithmetic agreement rather than physical accuracy.

C. An Example: SNRs in 1D Burgers’ Equation

  • The example uses a forward-time centered-space scheme with second-order spatial and first-order temporal accuracy, and a diffusive stability limit τ = O(h^2).Its central-difference advection term relies on physical viscosity for damping, making the scheme sensitive to grid-scale noise.Source: C. An Example: SNRs in 1D Burgers’… · p. 4
  • At practical resolutions h^-1 < 50, empirical SNR follows the O(h^-4) spatial discretization limit; only extreme over-refinement lets round-off overtake it.Temporal refinement improves SNR only until the spatial floor caps it.Source: C. An Example: SNRs in 1D Burgers’… · p. 4

D. Moving Beyond Arithmetic Baselines

  • Same-grid high-precision references share the solver’s discretization error, so their FP64-to-FP32 SNR gap measures arithmetic differences rather than physical accuracy.In this example, the reference curves show a gap that the analytical baseline does not.Source: D. Moving Beyond Arithmetic Baselines · p. 4, Figure 3 · p. 4
  • At h^-1 ≤50, FP32 and FP64 both reach ∼68 dB against the analytical solution, while only FP16 falls below that floor.The FP128-referenced curves instead show a large FP64-to-FP32 gap.Source: D. Moving Beyond Arithmetic Baselines · p. 4

E. Spectral Analysis and Stability

The spectral analysis relates round-off noise to the physical energy cascade and shows when grid-scale noise remains isolated or feeds back into resolved modes. The refinement analysis uses computable spatial or temporal differences to estimate a conservative discretization noise floor for precision checks.

E. Spectral Analysis and Stability

  • Round-off noise has a flat spectral floor scaling as ϵ2hd−2m; for a physical spectrum decaying as k^-α, their intersection defines a critical wavenumber beyond which the spectrum saturates.The elevated floor remains isolated at the truncation limit unless it intersects the cascade before dissipation sets in.Source: E. Spectral Analysis and Stability · pp. 4–5
  • At coarse resolution, the round-off floor meets the energy cascade after viscous damping; at fine resolution, it can intersect earlier and feed excess energy into resolved modes.The spectra illustrate this contrast: FP16 flattens near 30% of Nyquist at Nx = 128 and near 1% at Nx = 8192, where a tail curl occurs.Source: E. Spectral Analysis and Stability · p. 5, Figure 4 · p. 5

F. Empirical Estimation via Refinement

  • Because the exact solution is unavailable for complex PDEs, spatial or temporal refinement differences provide computable proxies for the true discretization error and conservatively bound its noise floor.The derivation expands the error into leading spatial and temporal truncation terms, then compares the target run with a proportionally refined run; the difference variance underestimates the true error variance.Source: F. Empirical Estimation via Refinement · p. 5
  • For the Burgers’ example, temporal and spatial refinement estimates bound the true floor from above across the discretization-limited range, but degrade beyond h^-1 ≈300 when difference fields become round-off dominated.The ordering is SNRtime ≥ SNRgrid > SNRtrue.Source: Figure 5 · p. 6
  • Safe precision reduction requires that its noise not push the signal below the full-grid noise floor (SNRgrid); a stricter temporal over-estimate is practical when full spatial refinement is impossible.Temporal-only refinement (rt > 1, rx = 1) is trivial in time-marching schemes and yields a practical black-box criterion for broad tiers such as FP64 versus FP32.Source: F. Empirical Estimation via Refinement · pp. 5–6

III. AUTOMATED PRECISION REDUCTION FRAMEWORK

The framework uses DaCe’s dataflow representation to lower precision in selected computational subgraphs without invasive source rewrites. It estimates a discretization noise baseline, verifies the lowered output against it, and reintegrates the optimized subgraph into the application.

III. AUTOMATED PRECISION REDUCTION FRAMEWORK

  • DaCe’s dataflow IR enables precision reduction on selected data containers, including intermediate transients, while automatically tracking downstream effects within a computational subgraph.This avoids invasive source-code rewrites.Source: §III · p. 6
  • The workflow extracts a target subcomputation, translates it to DaCe IR, and establishes a per-output discretization noise floor from coarse- and fine-resolution pairs or temporal refinement.Serialized reference data accompanies the extracted subcomputation.Source: §III · p. 6
  • The framework uniformly downcasts a target subgraph and validates it against the objective baseline, reporting failing fields if output SNR falls below the threshold.Uniform lowering avoids a combinatorial per-container search at the cost of optimality; successful subgraphs are compiled with glue code for reintegration.Source: §III · p. 6
  • GPU offloading is orthogonal to precision lowering; the framework runs offline, with initialization amortized across timesteps and no per-timestep cost beyond boundary casts.The stated overhead exception is the boundary casts themselves.Source: §III · p. 6

A. Type Semantics and Application Interfacing

The framework targets floating-point storage containers while relying on C++ type promotion for operations, and it manages precision transitions at isolated subgraph boundaries. Dataflow analysis can avoid casts for internal containers, while unavoidable casts add interface overhead.

A. Type Semantics and Application Interfacing

  • Lowering targets floating-point containers, including intermediate accumulators and reduction temporaries, while integer loop indices and control variables remain unchanged.Fully narrowed inputs run at reduced precision; mixed base- and lowered-precision operands are implicitly widened to base precision for intermediate operations.Source: A. Type Semantics and Application… · p. 6
  • An isolated lowered subgraph typically narrows inputs on entry and widens outputs on return, bounding its noise as a perturbation while leaving host memory state intact.These boundary casts are needed when the surrounding application remains at base precision.Source: A. Type Semantics and Application… · p. 6
  • Dataflow analysis eliminates casts for containers internal to the subgraph, but unavoidable boundary casts add per-invocation streaming passes comparable to a copy kernel.Widening the lowered scope removes these transitions; promotion overhead can also arise when hardware lacks native support for low-precision operations such as FP16 transcendentals.Source: A. Type Semantics and Application… · pp. 6–7

IV. EVALUATION

The evaluation tests whether precision lowering preserves physical fidelity and improves performance in two distinct weather-model kernels. It applies the automated DaCe pipeline on GPU systems and measures kernel performance under stated profiling conditions.

IV. EVALUATION

  • The evaluation asks whether reduced precision preserves physical fidelity and whether a safe tier yields performance gains.It establishes application noise floors, assesses FP64, FP32, FP16, and BF16, then measures validated configurations.Source: §IV · p. 7

1) Applications:

  • VT represents memory-bound unstructured-grid stencils, while CLOUDSC represents compute-bound column physics.VT is an ICON dynamical-core bottleneck; CLOUDSC advances cloud variables through microphysical conversions in independent vertical columns.Source: 1) Applications · p. 7
  • VT is evaluated at four grid spacings from 320 to 40 km, spanning 5,120 to 327,680 cells at 90 vertical levels.CLOUDSC uses a 120-second physics timestep; performance runs use 10 substeps, while SNR evolution uses 10,000.Source: 1) Applications · p. 7

2) System and Compilation:

  • Runs use GH200 and A100 GPU nodes, with precision lowering, GPU offloading, and boundary casts managed by the automated DaCe pipeline.Profiling isolates a single kernel step; CLOUDSC timings use the harness, VT timings come from full model runs, and all timings exclude boundary-cast overhead.Source: 2) System and Compilation · pp. 7–8, 3) Execution and Methodology · p. 8

3) Execution and Methodology:

The study judges precision against variance-SNR noise floors estimated through temporal and, for CLOUDSC, vertical refinement. FP32 clears the evaluated baselines, whereas lower precisions have application-, resolution-, and integration-dependent limits.

3) Execution and Methodology:

  • A target precision is safe only when its variance SNR remains above the discretization noise floor, estimated against refined configurations.The baseline uses temporal refinement for ICON-VT and temporal plus vertical refinement for CLOUDSC; vertical refinement is meaningful for CLOUDSC’s column structure.Source: 3) Execution and Methodology · p. 8, B. Numerical Accuracy · pp. 8–9

B. Numerical Accuracy

  • Some outputs are already noise-dominated at grid scale: FP64 floors are about 30 dB for VT vertical velocity and 12 dB for CLOUDSC PCLV.The safety test is whether lowered-output SNR stays above each field’s baseline, not whether its absolute error is small.Source: B. Numerical Accuracy · p. 9
  • FP32 is safe for both applications: CLOUDSC PT and PQ have SNRs of 124.2 dB and 116.2 dB, above their temporal and vertical grid baselines.All evaluated ICON-VT fields clear their SNRtime thresholds by wide margins; the SNR floors fall over the runs, while FP32 stays clear throughout.Source: Figure 7 · p. 8, B. Numerical Accuracy · p. 9
  • FP16 diverges to NaN for CLOUDSC PT and PQ; for VT, one variable must remain FP32, and admissibility is limited by resolution and integration length.At 160 km FP16 crosses the floor within about 200 steps; at 80 km and finer, the rising floor overtakes its noise level.Source: B. Numerical Accuracy · pp. 8–9
  • BF16 remains about 18 dB below FP16 across fields and resolutions; only w clears SNRtime at all four VT resolutions, and w_concorr_c only at 320 km.The result is consistent with BF16’s fewer significand bits relative to FP16.Source: B. Numerical Accuracy · p. 9

C. Computational Performance

FP32 provides substantial GPU speedups, while FP16’s additional gains are constrained by fidelity and hardware behavior. Field-specific refinement floors also show why a single user-set tolerance does not determine safe lowering uniformly.

C. Computational Performance

  • FP16 is fastest measured for VT, reaching 1.8× at 40 km on GH200 and 1.8× at 160 km on A100, but CLOUDSC is ruled out and the largest gains exceed the admissible range.At 160 km, FP16 is admissible only for short integrations and reaches 1.5× against FP32’s 1.3× on GH200; BF16 tracks FP16 speed within 0.1×.Source: C. Computational Performance · p. 9
  • At 80 km, VT DRAM reads fall from 1,850 MB in FP64 to 931 MB in FP32 and 483 MB in FP16, but bandwidth utilization drops to 23.7% in FP16.At FP16 the kernel is no longer constrained by main-memory bandwidth; roughly one-seventh of its floating-point work remains FP32, and BF16 retains nearly twice that share.Source: C. Computational Performance · pp. 9–10
  • 1.6–1.8× speedups on GH200 and 1.7–1.8× on A100 are reported for CLOUDSC FP32; VT reaches 1.6× on GH200 at 40 km.VT speedups range from 1.2× at 320 km to 1.6× at 40 km on GH200, and 1.3–1.5× on A100; halving element size halves DRAM traffic.Source: C. Computational Performance · p. 9, Figure 8 · p. 10

D. Field-Dependent Thresholds

  • RAPTOR emulates formats by truncating operations and compares outputs with FP64, while ADAPT ranks demotions against a user-specified absolute deviation bound.RAPTOR’s scope-based control is coarser than dataflow-based per-container lowering; ADAPT’s bound still leaves the tolerance choice to the user.Source: D. Field-Dependent Thresholds · p. 10
  • Across 17 CLOUDSC fields, discretization floors span 1.2–34.0 dB; FP32 falls below the floor for 3 fields, FP16 for 8, and BF16 for all 17.The thresholds are field-dependent, while RAPTOR’s predefined cutoff leaves the acceptability choice to the user.Source: D. Field-Dependent Thresholds · p. 10
  • ADAPT’s per-field error bounds span 24 dB across 16 fields, and the arrays it retains in FP64 differ by output field.At the selected bounds it leaves no arrays in FP64 for PCOVPTOT and PFHPSN, but 20 for PFSQLTUR.Source: D. Field-Dependent Thresholds · pp. 10–11

V. RELATED WORK

Prior weather and climate studies support precision reduction, but do not provide an automated scalar criterion for compiler-level tuning. The discussion positions refinement-based validation as complementary, while identifying assumptions and implementation boundaries.

V. RELATED WORK

  • Prior weather and climate studies validate precision reduction through ensemble or distributional comparisons, but lack an automated scalar criterion for compiler-level tuning.This work complements those studies with a principled threshold and an automated lowering framework.Source: §V · p. 11
  • Existing tuning tools leave tolerance open to users or compare against a same-grid high-precision reference; this framework evaluates against the physical discretization floor.The floor is measured by refinement and applies wherever a refined run is available; the cited analytical multigrid baseline is specific to SPD linear elliptic PDEs.Source: §V · p. 11
  • Sensitivity methods can be costly over chaotic integration windows, and adjoint estimates can understate incurred error when signed contributions cancel.The framework trades fine-grained optimality for scalable aggregate-output SNR validation against the discretization baseline.Source: §V · p. 11, §VI · p. 11

VI. DISCUSSION AND FUTURE WORK

  • Uniform downcasting lets one sensitive variable block precision reduction for an entire subgraph; intra-kernel mixing could recover those opportunities.Hardware without native low-precision operations can also lose bandwidth benefits to conversions; BF16 division and square root fall back to FP32.Source: §VI · pp. 11–12
  • The pseudo-quantization-noise assumption breaks when errors accumulate coherently or quantization noise swamps physical updates; empirical checks cover only tested states and windows.Longer horizons, broader ensembles, and formal stability criteria remain open coverage needs.Source: §VI · p. 12
  • The criterion assumes a discretization-limited solver, an available refined run, and a fluctuating output field with meaningful variance; without refinement, an arithmetic reference remains appropriate.The stated scope includes stable runs coarser than the resolution at which round-off would matter.Source: §VI · p. 12

VII. CONCLUSION

FP32 delivers up to 1.8× kernel speedups on both platforms, while the safe precision boundary varies by application and resolution. A variance-based SNR and automated DaCe framework target precision reduction, and the authors identify finer grids, higher-order schemes, or larger ensembles as uses for the recovered headroom.

VII. CONCLUSION

  • Over half of FP64's 52 significand bits are masked by discretization noise, which the authors formalize as a variance-based SNR in an automated DaCe framework.The framework downcasts dataflow subgraphs and validates the results.Source: §VII · p. 12
  • Up to 1.8× kernel speedups on both platforms, with FP32 clearing every baseline and approaching the theoretical 2× bandwidth ceiling.The results cover ICON velocity tendencies and CLOUDSC.Source: §VII · p. 12
  • FP16 is viable for ICON-VT in a sophisticated mixed-precision configuration but destructive when indiscriminately applied to CLOUDSC; BF16 clears neither application's floor.The safe precision boundary is application- and resolution-dependent, and the framework trades per-variable optimality for scalability.Source: §VII · p. 12
  • The bits made irrelevant by discretization are potential headroom for finer grids, higher-order schemes, or larger ensembles.Source: §VII · p. 12
Loading 2610.09089v1…