Source-linked AI summary
tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow
You Xie, Aleksandra Franz, Mengyu Chu, Nils Thuerey
TL;DR
The paper addresses the challenge of generating high-resolution fluid-flow data that remains temporally coherent without requiring temporal input sequences. It introduces a conditional volumetric GAN with a temporal discriminator and physics-aware training, demonstrating detailed, coherent flow synthesis from a single low-resolution time step.
Problem
Existing generative models do not account for temporal data evolution, while high-resolution turbulent-flow simulation is computationally expensive.
Method
A conditional volumetric GAN uses spatial and temporal discriminators, physics-aware augmentation, and additional physical quantities to infer high-resolution flow fields.
Results
The approach successfully learns four-dimensional flow-transport solutions, generating detailed and temporally coherent outputs from approximate solutions.
Takeaways & Limitations
The method provides instantaneous, reusable super-resolution for fluid data while offering basic artistic control over generated outputs.
Takeaways & Limitations
The network encodes a fixed resolution difference, and the current method may be slower than high-resolution advection; experiments have focused mainly on buoyant smoke clouds.
Abstract
from arXiv · showhide
We propose a temporally coherent generative model addressing the super-resolution problem for fluid flows. Our work represents a first approach to synthesize four-dimensional physics fields with neural networks. Based on a conditional generative adversarial network that is designed for the inference of three-dimensional volumetric data, our model generates consistent and detailed results by using a novel temporal discriminator, in addition to the commonly used spatial one. Our experiments show that the generator is able to infer more realistic high-resolution details by using additional physical quantities, such as low-resolution velocities or vorticities. Besides improvements in the training process and in the generated outputs, these inputs offer means for artistic control as well. We additionally employ a physics-aware data augmentation step, which is crucial to avoid overfitting and to reduce memory requirements. In this way, our network learns to generate advected quantities with highly detailed, realistic, and temporally coherent features. Our method works instantaneously, using only a single time-step of low-resolution fluid data. We demonstrate the abilities of our method using a variety of complex inputs and applications in two and three dimensions.
1 INTRODUCTION
The paper targets physically based, temporally coherent super-resolution for fluid flows, using a conditional GAN that infers high-resolution volumetric data from a single low-resolution frame. Its approach combines temporal discrimination, physics-aware training practices, and controllable physical inputs.
- Fluid-flow super-resolution is computationally expensive because resolving turbulent details with discretized simulations becomes infeasible at human space and time scales.
- The method performs instantaneous physically based interpolation from volumetric data at a single simulation time step, without additional temporal data or tracked quantities.
- Physics-aware data augmentation helps avoid overfitting while reducing the memory burden of large space-time physics datasets.
- A novel temporal discriminator complements the spatial discriminator to generate consistent, highly detailed results over time.
- The contributions include artistic control through additional loss terms and entangled physical input quantities, alongside an evaluation of adversarial training processes for physics functions.
- The authors present the approach as the first generative adversarial network for four-dimensional functions and demonstrate inference of flow-transport solutions from approximate solutions.
2 RELATED WORK
Related work spans deep generative modeling, image super-resolution, neural methods for numerical simulation, fluid up-sampling, and temporal coherence. The paper adapts these ideas to conditional inference of three-dimensional flow quantities while emphasizing temporally coherent generation.
- GANs generate complex data distributions conditionally or unconditionally, while this work uses a conditional architecture for high-resolution flow inference.
- Image super-resolution research established convolutional, residual, normalization, and perceptual-loss techniques that inform the network design and training choices.
- Prior temporal-generation methods either process sequences directly and generate results sequentially or add coherence losses between nearby frames.
- This method instead provides velocities to the networks, using them to support temporally coherent content generation through warping-related mechanisms.
- Earlier neural simulation methods targeted smaller two-dimensional problems, whereas this work focuses on conditional inference of high-resolution three-dimensional Navier–Stokes flow data.
- Fluid up-sampling and fluid-control methods share the goal of improving flow data and could be combined with this approach to generate coarse inputs with desired shape and timing.
3 ADVERSARIAL LOSS FUNCTIONS
tempoGAN combines conditional adversarial training with discriminator-based feature constraints and a temporal discriminator to produce detailed outputs whose evolution matches fluid-flow dynamics. Advection aligns consecutive frames before temporal judgment, improving coherence and training behavior over manually specified temporal losses.
- 3.1 Adversarial Training: The conditional GAN trains a generator and discriminator together, with the discriminator providing a learned loss that encourages generated outputs to match real data.The generator is trained to make its outputs acceptable to the discriminator, while both networks are optimized in turns.
- 3.2 Loss in Feature Spaces: Feature-space L2 constraints use discriminator features rather than a pre-trained vision network to control the generator for three-dimensional flow data.The discriminator is suitable because no appropriate pre-trained networks are available for three-dimensional flow problems.
- 3.2 Loss in Feature Spaces: Negative feature-loss weights can produce realistic, sometimes better results by encouraging features distinct from average reference features while remaining consistent with discriminator judgments.In the conditional setting, the inputs constrain the outputs, allowing feature variation without abandoning similarity to the desired data distribution.
- 3.3 Temporal Coherence: The temporal discriminator learns admissible temporal changes from data, complementing the spatial discriminator’s role in generating realistic details.It receives consecutive generated and real frames, with advection applied to align the neighboring frames before temporal evaluation.
- 3.3 Temporal Coherence: Advection-based temporal discrimination produces less jagged, less noisy, and more structured time derivatives than no temporal loss, an L2 temporal loss, or an unaligned discriminator.The full approach is reported as closer to ground truth, while the unaligned variant improves over L2,t but does not reach the quality of the advected version.
- 3.3 Temporal Coherence: Advection improves feature alignment and backpropagated gradients, helping the generator learn more coherent outputs even though it uses only approximate coarse velocity fields.When flow fields are unavailable, the unaligned temporal discriminator remains preferable to the simpler L2,t loss.
4 ARCHITECTURE AND TRAINING DATA
tempoGAN combines convolutional generator and discriminator designs with physics-based training data, augmentation, and adversarial optimization. Auxiliary physical fields and augmentation improve training quality while preserving physical structure and supporting efficient models.
- Network architecture: The model focuses on convolutional GAN architectures, including fully convolutional generators, U-nets, residual networks, and multi-layer discriminators.Fully convolutional networks can later process inputs of arbitrary sizes.
- Training data: Training uses randomized smoke simulations with down-sampled inputs and typically about 20 simulations producing 120 frames each.The simulations randomize smoke inflows, velocity inflows, and buoyancy forces.
- Input fields: Adding velocity to density substantially improves convergence and output quality, while vorticity provides subtler additional improvements over density-only inputs.The density-only generator still shows grid artifacts and lacks detailed structures after 40,000 iterations; the discriminator receives only density.
- Input fields: Physics-aware inputs are preferred over simply enlarging density-only networks because they shorten training, accelerate convergence, and improve long-term evaluation efficiency.Figure 7 reports that supplemental physical fields improve results even when the density-only version has twice as many weights.
- Data augmentation: Augmentation enables networks with millions of weights to train on only a few hundred samples without overfitting while respecting physical invariants such as Galilean invariance.Modified data are generated during training rather than requiring larger stored datasets.
- Data augmentation: Physics-aware augmentation transforms passive and directional fields appropriately while recomputing derived quantities such as vorticity.Velocity directions are adjusted under rotations and flips, whereas quantities involving mixed derivatives must be recomputed.
5 RESULTS AND APPLICATIONS
The method produces detailed, temporally coherent fluid-flow super-resolution results across two- and three-dimensional applications, while supporting velocity-based control. Experiments also characterize training behavior, performance, recursive use, and limitations.
- 3D Results: The method generalizes from down-sampled training inputs to regular simulation inputs without re-sampling while adding fine details closely resembling the input volumes.
- Applications: For obstacle-containing flows, outputs preserve small-scale features around the obstacle without leaking artifacts, from 256 × 180 × 180 inputs to 1024 × 720 × 720 outputs.The method is applied independently to each input volume in the sequence.
- Fine Tuning Results: Setting all velocities to zero suppresses small-scale detail generation, indicating that the network correlates velocity magnitudes with the amount of generated features.
- Additional Variants: Recursive application yields an 8× total resolution increase with plausible thin sheets, but tends to amplify features generated during the first application.
- Performance: Training typically takes around 14 hours in 2D and approximately 9 days in 3D, while generation scales linearly with volume cell count.Generating a 256^3 volume from a 64^3 input takes 17.9 seconds on average.
6 CONCLUSIONS
The paper presents a conditional GAN for four-dimensional data that preserves temporal coherence through a novel time discriminator. The authors report detailed, stable outputs, flexible sizing, and basic art-direction capabilities.
- The novel temporal discriminator enables generators to preserve temporal coherence for four-dimensional data and improves stable-detail generation and learning.
- Fully convolutional networks accept inputs of arbitrary size, while additional physical inputs provide basic means for art direction.
- The authors identify applying tempoGAN to other physical problems or non-physical data such as video streams as future work.
A DETAILS OF NN ARCHITECTURES
The architecture description defines notation for input and output tensors, interpolation, convolutions, residual blocks, and spatial resolution changes in two- and three-dimensional networks.
- The notation specifies input and output resolutions and channels, nearest-neighbor interpolation, convolutional layers, and residual blocks.
- The same resolutions and filter sizes are used across spatial dimensions, and feature-map resolutions decrease when convolution strides exceed one.
B PARAMETERS & DATA STATISTICS
The appendix summarizes training and data-generation parameters, model-size accounting, repeated network updates per iteration, and standard-model annotations.
- The parameter summary includes training-run and data-generation settings, with model size reported after compression.
- Networks are trained multiple times per iteration according to the listed training configuration.
- Standard models used unless otherwise indicated are marked with an asterisk in the training-run table.