Source-linked AI summary
Dichoptic Foveation
Henry Kam, Colin Groth, Jenna Kang, Pratham Saraf, Qi Sun, Kenneth Chen
TL;DR
The paper addresses the limited understanding of how interocular frequency differences are perceived across the visual field. It measures dichoptic foveation psychophysically, fits a perceptual model, and applies it to VR foveated rendering, where the method supports roughly 50% sampling-rate reductions at perceptually equivalent quality.
Problem
Perception of interocular frequency differences and blur–sharpen interactions across retinal eccentricities had not been studied, despite their relevance to binocular fusion and stereoscopic displays.
Method
The authors collected psychophysical judgments of blurred and sharpened images presented to opposite eyes at different eccentricities, scaled them to JODs, and fit a model over blur, sharpening, and eccentricity.
Results
~50% sampling-rate reductions were achieved versus traditional foveated rendering while maintaining perceptually equivalent quality to full-resolution renderings.
Takeaways & Limitations
Dichoptic foveation provides a compact perceptual model for linking binocular rendering parameters to perceived quality in VR.
Takeaways & Limitations
The model averages across observers and does not explicitly account for individual eye dominance, which may affect perceived quality and dichoptic-foveation effectiveness.
Abstract
from arXiv · showhide
Interocular differences in visual perception can induce a variety of effects when fused by the brain. For example, prior works have found that carefully crafted binocular differences in local detail can improve contrast. It has also been found that when the frequency content of two stimuli are slightly different, blur suppression leads to a fused percept that is typically dominated by the sharper image. In this paper, we develop a psychophysical framework to measure the perception of natural image stimuli with interocular frequency differences across the visual field. To this end, we study the effect of dichoptic foveation, which we define as the application of blur to one eye and a simultaneous sharpening filter to the other. Stimuli were viewed in a virtual reality (VR) head-mounted display (HMD) and placed at different retinal eccentricities. Study data were scaled to a perceptual just objectionable difference (JOD) scale, and a 4D model was fit to it; our results suggest that interocular frequency differences can be well described by a simple computational model. We applied the model in a realistic VR scenario with free exploration of 360° videos to improve a base foveated rendering system by enhancing high frequency information dichoptically.
1 Introduction
The paper introduces dichoptic foveation to study how blurred and sharpened images presented to opposite eyes interact across retinal eccentricities. A psychophysical model predicts perceived quality and supports VR foveated rendering with substantially reduced sampling.
- Motivation: No prior work had studied how interocular frequency differences and blur–sharpen interactions are perceived across the visual field.Earlier work showed blur suppression and benefits from crafted binocular differences, but not their variation with retinal eccentricity.
- Approach: Dichoptic foveation presents a blurred stimulus to one eye and a sharpened image to the other, varying the configuration with retinal eccentricity.Retinal eccentricity is defined as angular distance from the fovea.
- Approach: 12,989 human-subject trials measured perception of dichoptic stimuli, and a three-variable model predicted quality in just-objectionable differences using blur, sharpening, and eccentricity.The study developed a psychophysical framework for natural-image stimuli viewed in VR.
- Application: ~50% compute savings were demonstrated at similar perceptual quality by applying dichoptic foveation to foveated rendering.The comparison was calibrated for minimal perceptual impact relative to full-resolution reference content.
2 Background & Related Work
The background frames dichoptic perception as binocular fusion of unequal visual inputs, including blur suppression and rivalry when differences grow. Related graphics work has used binocular differences and eccentricity-dependent rendering, but not frequency differences across the visual field.
- Binocular and Dichoptic Perception: Binocular fusion integrates the two eyes’ retinal images into a unified percept within their overlapping visual field.The forward-facing binocular field spans approximately 110° horizontally, while each eye also has a non-overlapping monocular periphery.
- Binocular and Dichoptic Perception: Small sharpness differences can preserve fusion, sharper images can dominate through blur suppression, and sufficiently large discrepancies can produce binocular rivalry.Fusion and rivalry may coexist in a tristable regime spanning 25°–35°.
- Dichoptic Rendering in Computer Graphics: Prior dichoptic graphics methods improved perceived contrast, supported gaze guidance, or reduced computation through unequal tonemapping, focus cues, or compression rates.These approaches did not study dichoptic frequency differences across the visual field.
- Foveated Rendering: Foveated rendering exploits declining visual acuity with retinal eccentricity by reducing rendering samples farther from the gaze point.Related uses include display-power reduction, stereo-perception improvements, and cybersickness reduction.
3 Psychophysical Framework
The psychophysical framework measures natural-image perception under asymmetric blur and sharpening across retinal eccentricities. Pilots selected perceptually spaced parameters, and a within-subjects study quantified image quality using JOD scores.
- 3.1 Method: Blur and sharpening were modeled as Gaussian-filter operations, with unsharp masking adding a scaled high-frequency residual to the original image.The same Gaussian standard deviation was used for both operations, although sharpening was not an exact inverse of blurring.
- 3.1 Method: The study used twelve natural Pixabay image patches spanning low, medium, and high spatial-frequency content, displayed through a Meta Quest Pro HMD.Patches were categorized using relative high-frequency energy in their Fourier power spectra.
- 3.3 Pilot Experiments: Two pilots established a reasonable blur–sharpen range and selected five approximately evenly spaced blur and sharpening levels at 0.5-JOD decrements.The selected levels were Blur σ: [0, 0.2704, 0.5408, 0.8113, 1.0817] and Sharpen ϕ: [0, 0.2235, 0.4471, 0.6706, 0.8941].
- 3.4 Main Study: 31 participants compared dichoptic test stimuli with binocular unaltered references in a 2AFC task across eccentricity, blur, sharpening, and image content.Participants freely toggled between the reference and two test stimuli, with gray blanks preventing direct comparisons.
- 3.5 Results: Blur, sharpening, and eccentricity each had strong main effects, while eccentricity significantly modulated blur and sharpening effects but blur–sharpen interaction was nonsignificant.The reported p-values were 5.44 · 10^-8, 6.32 · 10^-8, and 5.77 · 10^-16 for the three main effects; blur–sharpen interaction p=0.707.
4 Computational Model
The paper fits four-dimensional perceptual models that predict JOD impact from blur, sharpening, and eccentricity. A linear model extrapolated most accurately beyond the experimental parameter range, especially at higher eccentricities.
- 4 Computational Model: Four-dimensional models predict JOD perceptual impact from blur, sharpening, and eccentricity using linear, interaction, exponential, and quadratic parameterizations.The quadratic model was used to interpolate the empirical data and visualize model surfaces by eccentricity.
5 Application to Foveated Rendering
The perceptual model is applied to dichoptic foveated rendering to preserve visual quality while reducing sampling requirements. The system uses eccentricity-dependent blur and sharpening in a realistic VR setting with dynamic gaze considerations.
- 5 Application to Foveated Rendering: Dichoptic foveated rendering demonstrates potential for approximately 50% compute savings at similar perceptual quality.The comparison uses blur magnitude as a proxy for system-agnostic computational cost, and quality and sampling objectives were validated separately.
- 5 Application to Foveated Rendering: The linear model selects blur and sharpening parameters because its planar objective surface avoids local minima and extrapolates robustly across wide-FOV HMD eccentricities.This permits analytic optimization rather than iterative optimization solvers.
- 5 Application to Foveated Rendering: The rendering strategy maximizes blur magnitude because blur reduces required samples, while sharpening has constant O(1) computational overhead.Sharpening is implemented with a single addition and multiplication, making its cost negligible relative to rendering costs.
- 5 Application to Foveated Rendering: Sharpening gain can decrease with eccentricity to permit greater peripheral blur, while foveal sharpening remains higher to preserve quality and prevent binocular rivalry.The tested decreasing-gain condition reached ϕ=0.22 between 0° and 60° eccentricity, with blur sampled at −1 JOD.
- 5 Application to Foveated Rendering: The application complements classic foveated rendering by applying dichoptic sharpening to the low-pass information remaining after foveation.Baseline parameters were calibrated for a dynamic gaze setting, and foveation intensity was calibrated with 14 participants viewing three stereoscopic 360° videos.
5.3 Subjective Quality Evaluation
Subjective VR evaluation found dichoptic foveated rendering can preserve perceptual quality while reducing sampling requirements, including without eye tracking.
- 5.3 Subjective Quality Evaluation: The VR validation used ten stereoscopic 360° videos, 14 participants, randomized rendering modalities, and ten-second freely explored trials.Participants rated each video afterward on a 5-point Likert scale conforming to ITU-T P.910.
- 5.3 Subjective Quality Evaluation: Dichoptic rendering with dynamic sharpening and standard foveated rendering both matched the reference video in visual quality, whereas constant sharpening differed significantly.Mean MOS values were 3.26 for both dynamic sharpening and standard foveated rendering, versus 3.66 for the reference; constant sharpening scored 3.16.
- 5.5 Computational Benefit: Dichoptic foveation achieved similar visual quality to full-resolution rendering despite significant blur, while larger local sample spacing represented greater computational savings.The analysis relates sampling interval and blur through a Gaussian point-spread-function assumption.
- 5.5 Computational Benefit: For the evaluated VR HMD, dichoptic foveation required about half the samples of traditional foveated rendering to maintain comparable perceived quality.Theoretical workload analysis yielded a benefit of approximately 1.9; smaller AR displays yielded theoretical savings up to 11×.
- 5.6 Ablation: Dichoptic Rendering Without Eyetracking: Uniform dichoptic rendering reduced theoretical sampling requirements by approximately 97% relative to native-resolution rendering, but quality depended on which eye was blurred.With the left eye blurred, MOS was 3.63 versus 4.19 for the reference without a significant difference after correction; right-eye blurring differed significantly from the reference.
6 Limitations & Future Work
The paper’s conclusions are bounded by observer variability, hardware and binocular-overlap constraints, unexplored long-term effects, and theoretical rather than hardware-specific savings.
- 6 Limitations & Future Work: The model targets a typical observer, so individual sensitivity and binocular-rivalry differences may require per-observer calibration.The authors identify per-observer calibration as a future direction for improving subject-level performance.
- 6 Limitations & Future Work: The model does not explicitly account for eye dominance or dynamic effects, although ablation results suggest effectiveness may depend on which eye receives the sharper signal.Eye-dominance-aware rendering is proposed for more personalized dichoptic foveation.
- 6 Limitations & Future Work: Experiments used a Meta Quest Pro, whose eye-tracking latency, resolution, and optical distortions may introduce artifacts unrelated to the studied phenomena.The authors nevertheless describe the calibration conditions as representative of realistic VR viewing.
- 6 Limitations & Future Work: Dichoptic foveation works only in binocular-overlap regions, so high-FOV headsets may yield less savings and require standard foveated rendering at monocular edges.Normal foveated rendering can supplement the technique in the outer monocular regime.
- 6 Limitations & Future Work: The evaluation did not examine long-term discomfort from eye strain or binocular rivalry, which remains future work for daily-use scenarios.The application-focused study assessed perceptual quality relative to a baseline reference condition.
- 6 Limitations & Future Work: The reported computational savings are theoretical and system-agnostic, so real raytracing optimizations may produce different hardware-specific results.The authors call for implementation on an actual system to measure realized savings.
7 Conclusion
Dichoptic foveation uses binocular fusion to optimize stereoscopic rendering while preserving perceived quality. Its perceptual model enables approximately 50% sampling-rate reductions relative to traditional foveated rendering.
- Dichoptic foveation parameterizes blur and sharpening by retinal eccentricity to link rendering settings with perceived quality.
- ~50% sampling-rate reductions are enabled in VR while maintaining perceptually equivalent quality to full-resolution renderings.