Source-linked AI summary
Animating Pictures with Eulerian Motion Fields
Aleksander Holynski, Brian Curless, Steven M. Seitz, Richard Szeliski
TL;DR
A still image does not directly provide motion, so the paper targets scenes with fluid motion that can be represented by a static Eulerian field. It estimates that field from the image, animates deep features with it, and reports plausible motion and realistic animations across a wide collection of images.
Problem
The paper addresses how to synthesize plausible motion from a still image, focusing on fluid scenes that are well approximated by particles moving through a static velocity field.
Source: §1 · p. 1Method
An image-to-image network predicts an Eulerian motion field, which is integrated to animate the image through symmetric feature splatting and video looping.
Source: §1 · p. 2, §4 · p. 3, §5.1 · p. 4Results
The method produced plausible motion and realistic animations across a wide collection of images with fluid motion.
Source: §8 · p. 8Takeaways & Limitations
The paper presents single-photograph animation into looping videos using Euler-integrated motion and symmetric splatting that combines texture from the future and past.
Source: §8 · p. 8The method restricts its focus to fluid motions, such as smoke, water, and clouds, that are well approximated by particle motion through a static velocity field.
Source: §1 · p. 1
Abstract
from arXiv · showhide
In this paper, we demonstrate a fully automatic method for converting a still image into a realistic animated looping video. We target scenes with continuous fluid motion, such as flowing water and billowing smoke. Our method relies on the observation that this type of natural motion can be convincingly reproduced from a static Eulerian motion description, i.e. a single, temporally constant flow field that defines the immediate motion of a particle at a given 2D location. We use an image-to-image translation network to encode motion priors of natural scenes collected from online videos, so that for a new photo, we can synthesize a corresponding motion field. The image is then animated using the generated motion through a deep warping technique: pixels are encoded as deep features, those features are warped via Eulerian motion, and the resulting warped feature maps are decoded as images. In order to produce continuous, seamlessly looping video textures, we propose a novel video looping technique that flows features both forward and backward in time and then blends the results. We demonstrate the effectiveness and robustness of our method by applying it to a large collection of examples including beaches, waterfalls, and flowing rivers.
1. Introduction
People can infer motion from a still scene, and this paper learns natural-scene motion priors to animate a new image. It focuses on fluid motion, which is well approximated by particles moving through a static velocity field.
1. Introduction
- The method learns motion priors from real-scene videos to synthesize plausible motion for a novel static image.It predicts an Eulerian motion field from the image, then uses deep feature warping and a looping technique to render an animated video.Source: §1 · p. 1
- The method targets fluid motion such as smoke, water, and clouds, modeled as particle motion through a static velocity field.General scene motion is more complex because it involves perspective effects, occlusions, and transience.Source: §1 · p. 1
- The paper contributes Euler-integrated motion, symmetric deep splatting for frame synthesis, and a technique for seamless looping.Together, these components produce looping animations from a single image.Source: §1 · pp. 1–2
2. Previous Work
Prior video-texture methods create looping clips from video input, while single-image animation methods use manual motion, simulations, guidance videos, or learned priors. The paper situates its approach among these alternatives and compares it with Endo et al. [6].
2.1. Video Textures
- Video-texture methods produce seamless or infinite loops from longer video sequences, but require video input.Cinemagraphs may animate only user-selected regions; automatic methods identify regions suitable for looping.Source: §2.1 · p. 2
2.2. Single-image animation
- Single-image animation alternatives include user-painted motion, physical simulation, reference-video guidance, and latent-space transformations.Simulation depends on identifying objects and their extent; some video-guidance methods require segmentation of the dynamic region.Source: §2.2 · p. 2
- Latent-space animations can vary plausible appearances without necessarily producing plausible motion, and spatial warps require feasible scene motion.Video prediction methods typically predict multiple future frames at once, limiting them to low resolution or few frames.Source: §2.2 · p. 2
- Compared with Endo et al. [6], the paper reports more reliable fluid-scene motion estimation and fewer visible animation artifacts.Endo et al. [6] synthesize motion and appearance for animating timelapses from static landscape imagery.Source: §2.2 · p. 2
3. Overview
Given one static image, the pipeline estimates motion, uses it to animate the image, and produces a video whose start and end frames match.
3. Overview
- An image-to-image translation network estimates a motion field from the input image to define pixel positions in future frames.The pipeline then animates the image through deep warping.Source: §3 · p. 2
- Deep warping uses the estimated source-pixel positions to animate the input image.Source: §3 · p. 2
- A looping technique ensures that the generated video starts and ends with the same frame.Source: §3 · p. 2
4. Motion estimation
The method estimates a single static Eulerian motion field from an input image and integrates it to obtain pixel trajectories. It predicts forward motion, which the paper reports improves motion reliability and boundary sharpness relative to backward-flow prediction.
4. Motion estimation
- The overview shows the estimated field being integrated into future and past displacements, which warp encoded features before decoding them into video frames.Source: Figure 2 · p. 3
- A static Eulerian field assigns each pixel an immediate velocity that remains constant over time, while Euler integration can still produce complex motion paths.The method predicts the field once from the input image rather than recurrently estimating incremental flows, avoiding the long-term distortion associated with dependent predictions.Source: §4 · p. 3
- An image-to-image translation network trained on color-motion pairs estimates a plausible motion field once for a new image.The field is then recursively applied to define source-pixel positions in future frames, without further network calls.Source: §4 · p. 3
- Forward-motion prediction yields more reliable motion estimates and sharper object boundaries, supporting more realistic animation of nearby scenes with partial occlusions.The paper contrasts its input-aligned forward motion field with Endo et al.'s [6] backward-flow prediction.Source: §4 · p. 3