Source-linked AI summary
Masking Judder in Foveated Rendering with Phase-Shifted Noise
Ville Cantory, Danhua Zhang, Victoria Interrante, Piotr Didyk
TL;DR
Reduced temporal rendering rates can save computation but cause motion judder, while filtering can suppress judder at the cost of detail. This paper advances synthesized Gabor noise between rendered frames using phase shifts and selects its frequency to mask judder without aliasing. Experiments report higher tolerated motion velocities and no significant image-quality loss.
Problem
The paper asks whether synthesized noise can reduce the visibility of judder from reduced temporal resolution beyond what filtering achieves, which otherwise costs computation and detail.
Source: §1 · p. 2Method
Results
1.96× over full resolution rendering at high foveation and 1.66× at low foveation: adaptive noise raised tolerated camera-motion velocity, while validation found no significant image-quality loss.
Source: §1 · p. 2, §6 · p. 10Takeaways & Limitations
Advancing noise between rendered frames can limit judder visibility at reduced temporal rates while avoiding full-scene rendering on every display frame.
Source: §1 · p. 2, §8 · p. 11The work does not develop a general model to automatically tune the configuration for optimal judder reduction and image quality.
Source: §7 · p. 11
Abstract
from arXiv · showhide
Low framerates during rendering arise when frame time exceeds the rendering budget, or can be introduced deliberately to save computation, for example in foveated rendering, where the periphery can be rendered at a reduced spatial and temporal resolution. However, low frame rates also cause non-smooth motion, or judder, degrading perceived image quality. Judder manifests as aliasing in the spatiotemporal frequency domain; temporal filtering, such as blurring, can suppress it, but at an additional computational cost and incur a larger loss of detail. Prior work has shown that inexpensive synthesized noise can compensate for the loss of spatial detail and velocity perception caused by foveation, but its potential for restoring motion cues lost to reduced temporal resolution has not yet been explored. We show that such noise can itself restore these cues: translating a Gabor kernel reduces to a phase shift, and modulating a kernel phase creates a perception of smooth motion without having to fully rerender the noise. We derive a method that, given the on-screen angular velocity and the temporal rendering rate, selects a spatial frequency noise band such that phase shifts perceptually mask judder without introducing motion aliasing of their own. User experiments show that our technique substantially expands the range of angular velocities over which motion at reduced temporal resolution remains smooth, without objectionable degradation of image quality.
1. Introduction
Foveated rendering saves computation by reducing peripheral detail, but reduced temporal resolution can make motion visibly judder. This paper investigates whether synthesized noise can mask judder while preserving detail and motion cues.
1. Introduction
- Foveated rendering reduces peripheral sample density to save shading cost, but removing detectable spatial frequencies can degrade perceived image quality.Lower rendering rates can also produce judder, which becomes more objectionable at higher velocities.Source: §1 · p. 2
- Phase-shifted noise advanced at the full display rate substantially limits judder visibility when peripheral content is rendered at a reduced rate.The paper describes the intermediate noise updates as less costly than rendering the full scene or applying temporal filtering.Source: §1 · p. 2
- The method advances synthesized noise between rendered frames by phase-shifting translated Gabor kernels, without resynthesizing the noise.The approach reconstructs orientation-aligned local first- and second-order statistics from the foveated image.Source: §1 · p. 2
- Experiments report smoother perceived motion at higher velocities than foveation without noise synthesis, while the derived model selects a noise frequency intended to avoid aliasing.The validation experiment reports substantial judder masking without significant image-quality loss.Source: §1 · p. 2
2. Related Work
The related work spans motion artifacts, foveated rendering, image metamers, and temporal reprojection. This paper builds on these areas by using already synthesized noise to provide motion cues between rendered frames.
2.1. Motion Artifacts
- Motion judder is visible discretization of continuous motion; motion blur can hide it but may add computation and remove high-frequency detail.The discussion distinguishes judder from hold-type blur, which affects motion clarity at lower refresh rates.Source: §2.1 · p. 2
2.2. Foveated Rendering
- Foveated rendering reduces peripheral sample density to exploit lower visual acuity, and this paper models reduced resolution with a spatially varying Gaussian blur.The noise synthesis and subsequent phase shifts take the foveated image as input.Source: §2.2 · p. 3
2.3. Metamerism
- Peripheral metamers match local image statistics rather than true pixel colors; prior Gabor-noise methods restore spatial detail or lost velocity perception.Unlike those methods, this work modulates already constructed noise in intermediary frames using estimated motion velocity and phase shifts.Source: §2.3 · p. 3
2.4. Temporal Reprojection
- Temporal reprojection reuses rendered frames to avoid rendering from scratch, while this method aims to mask judder at low render cost and complement reprojection.Figure 2 illustrates that the same phase step displaces the sinusoid less at higher spatial frequencies, so lower-frequency Gabors are needed to mask ρJ.Source: §2.4 · p. 3, Figure 2 · p. 3
3. Noise Synthesis
The method synthesizes peripheral Gabor noise from the foveated image, matching local image statistics to restore texture energy without creating new structures. Frequency, orientation, amplitude, and phase determine the noise and how it can convey motion.
3. Noise Synthesis
- Gabor noise synthesized from the foveated frame restores detectable but unresolvable peripheral texture energy by matching local image statistics without creating new visual structures.The base noise is formed by convolving Gabor kernels with sparse random impulses.Source: §3 · p. 3
- Synthesized frequencies lie between eccentricity-dependent peripheral resolution and detection limits, while scale s lowers both band bounds to support motion cues.The band is constrained further by the foveation blur response.Source: §3 · pp. 3–4
- Gabor orientation follows local image structure for static scenes, but aligns with motion flow to maximize motion conveyed by phase shifting.The motion-flow structure is estimated from surviving low frequencies and is spatiotemporally stable.Source: §3 · p. 4
- Noise amplitude is inferred from the Laplacian-pyramid band nearest the local foveation cutoff, with a global gain setting its strength.The synthesis is added to luminance with contrast enhancement, and amplitude is attenuated against local image bounds to prevent clipping.Source: §3 · p. 4
- Phase is the only Gabor parameter modulated to advance noise at the full display rate and convey motion.This advances the kernel without resynthesizing the noise.Source: §3 · p. 4
4. Exploratory Perceptual Study
The exploratory study tested whether synthesized noise could reduce perceived judder when low-rate rendered content remained static between frames. Noise conditions supported higher matched velocities than blur, and lower noise scales generally extended that range.
4. Exploratory Perceptual Study
- At 30Hz, matched velocity rose from 31.0 deg/s under blur to 51.1–59.1 deg/s across noise scales; at 20Hz, from 8.9 to 18.7–33.9 deg/s (1.65–3.8×).The result was pooled over eccentricities and textures.Source: Figure 4 · p. 4, §4 · p. 5
- Even when low-frequency image content stayed static between rendered frames, synthesized noise reprojection along motion flow conveyed motion; lower noise frequencies represented faster motion under the same phase step.Lower frequencies produce greater displacement between frames.Source: §4 · pp. 4–5
- Five graphics experts adjusted sphere velocity against a 60Hz foveated reference across 60/30/20Hz test rates and three noise scales at eccentricities of 18°, 24°, and 30°.The stimuli were textured spheres; the test side used blur alone or noise at s = 1, 0.75, or 0.5.Source: §4 · pp. 4–5
- Noise improved matched smoothness over blur at every rendering rate, generally with higher matches at lower scales, no texture difference, and a partial-trend exception at 24° and 30Hz.The authors suggest the exception may reflect the eccentricity-dependent peak of peripheral contrast sensitivity.Source: §4 · p. 5
5. Phase Shifts Mask Motion Judder
The section models judder as visible spatiotemporal aliasing and develops velocity-adaptive phase-shifted noise to carry motion while avoiding perceptible noise aliasing.
5. Phase Shifts Mask Motion Judder
- Noise frequency must adapt to motion: lower frequencies extend phase-shift masking to faster motion, but can become distracting or resolvable.The method selects the highest frequency whose phase shifts can still convey the motion.Source: §5 · p. 5, §5.2 · p. 6
5.1. Motion Judder
- Sampled motion creates temporal aliasing copies; higher angular velocities pull them toward more visible frequencies, where judder visibility depends on aliased energy and contrast sensitivity.The section uses a contrast-sensitivity model to predict visibility as the boundary shifts with eccentricity.Source: §5.1 · pp. 5–6
5.2. Visibility of Judder and Noise
- At 20Hz and 12Hz, predicted critical velocities are roughly 4–30 deg/s across 28–42° eccentricity; lowering noise scale raises predicted sensitivity.Lower frequencies let the same phase step convey faster motion but may make the noise visible.Source: §5.2 · p. 6
5.3. Phase Conveys Motion
- The method chooses the maximum frequency scale whose phase step stays within the motion-aliasing constraint, using the noise update rate rather than the content rate.The scale depends on the synthesis band, noise rate, and on-screen angular velocity; the noise’s own aliasing copy remains outside the visibility boundary.Source: §5.3 · pp. 6–8
- Translating a sinusoidal component is implemented as a phase advance, so seeded noise impulses can stay fixed while their sinusoidal detail drifts.The phase shift is the discrete counterpart of continuous phase progression.Source: §5.3 · pp. 6–7
5.4. Implementation & Performance
- Noise synthesis, rescaling, and phase shifting average 2.3 ms at 4K (~430 fps) on an RTX 5070; foveation was simulated with Gaussian blur.The authors note that true shading-rate reduction could further improve performance.Source: §5.4 · p. 8
6. Validation Experiment
The validation study found that phase-shifted noise let viewers tolerate higher angular velocities before perceiving judder, while image-quality effects depended on foveation level. The experiment tested noise rendered at 60 Hz over scene content rendered at 20 or 12 Hz.
Setup
- The study tested maximum right-edge eccentricities of 28, 35, and 42 degrees and clamped adaptive frequency scale to s ≥ 0.08.The low foveation level was tuned just below visibility, and the high level was set 50% higher.Source: §6 · p. 8
Task
- The study compared low- and high-foveation rendering across Forest and City scenes, with underlying scene content rendered at 20 Hz or 12 Hz and noise always rendered at 60 Hz.Participants adjusted camera velocity until motion appeared smooth, then rated peripheral rendering distraction on a scale from 1 to 5.Source: §6 · p. 9
Results
- At high and low foveation, adaptive noise raised tolerated velocity 1.96× and 1.66× over full-resolution rendering, and 1.62× and 1.56× over matched blur.The comparisons are pooled over scenes and eccentricities; Figure 12 shows adjusted angular velocities by scene, rendering condition, rate, and maximum eccentricity.Source: Table 1 · p. 9, Figure 12 · p. 10, §6 · p. 10
- At low foveation, noise distraction ratings did not differ from matched blur (p = 0.83); at high foveation, ratings were 0.67 higher than matched blur (p = 0.004).Within noise conditions, ratings rose with adjusted velocity (median within-participant Spearman ρ = 0.15, p = 0.007).Source: §6 · p. 10, Figure 13 · p. 11
- At high velocities, adaptive scaling lowers noise frequency to avoid motion aliasing, but the resulting structure can become resolvable and distracting, especially with stronger foveation.Low foveation leaves more room to lower the noise frequency while keeping it above resolvable frequencies; at high foveation, more missing spatial spectrum must be replaced.Source: §6 · pp. 8, 11
7. Limitations & Future Work
The method specifies noise synthesis and phase advancement for chosen foveation, velocity, and rendering rate, but does not automatically optimize that configuration. The authors identify further work on joint configuration selection, efficiency, cybersickness evaluation, and measuring judder reduction at fixed image quality.
Limitations
- The method does not provide a general model to automatically tune noise configuration for the trade-off between judder reduction and image quality.The specified configuration depends on foveation level, velocity, and rendering rate; future work could jointly select these with viewing conditions, scene content, and rendering budget.Source: §7 · p. 11
- Noise-generation efficiency could be improved further, although caching is already used to optimize frame-to-frame Gabor-noise phase shifts.Source: §7 · p. 11
Future Work
- The authors propose evaluating whether the noise may help mitigate cybersickness in VR; this remains future work.They note that prior work on high-frequency noise and countervection cues examined full-temporal-resolution displays.Source: §7 · p. 11
Limitations
- The study measures the trade-off at the highest angular velocity judged smooth, rather than measuring velocity when image quality begins to degrade.The authors suggest a study that stops adjustment when quality degrades to assess judder mitigation at the same image quality across conditions.Source: §7 · p. 11
8. Conclusion
The method combines foveated rendering at reduced frame rates with noise synthesized at higher frame rates and advanced through phase shifts. In validation, it increased tolerated angular velocity, without added distraction relative to blur at the low foveation level.
Conclusion
- Phase-shifted noise at higher frame rates limited judder visibility while the underlying foveated scene remained at a reduced frame rate.Source: §8 · p. 11
- Intermediate frames require only noise phase shifting and rescaling, rather than rendering the full frame.Source: §8 · p. 11
- At a foveation level just below visibility, viewers tolerated higher angular velocities without increased distraction relative to blur alone.Source: §8 · p. 11