Source-linked AI summary
Perceptual Quality Assessment of Omnidirectional Images
Huiyu Duan, Guangtao Zhai, Xiongkuo Min, Yucheng Zhu, Yi Fang, Xiaokang Yang
TL;DR
The paper studies omnidirectional image quality assessment for VR, where viewers see only part of a scene and viewport visual attention differs from traditional imagery. It constructs the OIQA database, collects subjective ratings with head and eye movements, and evaluates IQA models, finding both shared performance patterns and VR-specific deviations.
Problem
Omnidirectional IQA lacks a database combining subjective quality ratings with visual-attention data, despite viewers seeing only part of scenes in VR.
Method
The authors build OIQA with 16 raw and 320 distorted images, conduct VR quality ratings while tracking head and eye movements, and evaluate nine IQA models using PLCC, SRCC, and RMSE.
Results
FSIM, GSI, and VSI perform best overall, while many other IQA models drop in performance on omnidirectional images; WGN ratings are consistently higher than model predictions.
Takeaways & Limitations
The study suggests that VR viewers prefer high-frequency content and image details, making omnidirectional IQA different from traditional IQA in some cases.
Abstract
from arXiv · showhide
Omnidirectional images and videos can provide immersive experience of real-world scenes in Virtual Reality (VR) environment. We present a perceptual omnidirectional image quality assessment (IQA) study in this paper since it is extremely important to provide a good quality of experience under the VR environment. We first establish an omnidirectional IQA (OIQA) database, which includes 16 source images and 320 distorted images degraded by 4 commonly encountered distortion types, namely JPEG compression, JPEG2000 compression, Gaussian blur and Gaussian noise. Then a subjective quality evaluation study is conducted on the OIQA database in the VR environment. Considering that humans can only see a part of the scene at one movement in the VR environment, visual attention becomes extremely important. Thus we also track head and eye movement data during the quality rating experiments. The original and distorted omnidirectional images, subjective quality ratings, and the head and eye movement data together constitute the OIQA database. State-of-the-art full-reference (FR) IQA measures are tested on the OIQA database, and some new observations different from traditional IQA are made.
I. INTRODUCTION
The paper addresses omnidirectional image quality assessment for immersive VR, where viewers see only part of a scene and visual attention differs from traditional imagery. It introduces the OIQA database and evaluates quality-assessment methods using subjective ratings and attention data.
- VR omnidirectional content can provide immersive perception of real-world scenes, motivating quality assessment in head-mounted displays.
- Existing omnidirectional-content studies and traditional IQA databases do not provide an IQA database combining subjective ratings with visual-attention data.
- Viewers frequently see only a portion of an omnidirectional scene, and viewport saliency differs from saliency in traditional 2-D images and videos.
- OIQA contains 16 raw omnidirectional images and 320 degraded images covering JPEG, JPEG2000, Gaussian blur, and Gaussian noise.
- The study tracks HMD view direction and eye movements during subjective evaluation, releases the resulting database, and evaluates state-of-the-art IQA models.
II. SUBJECTIVE QUALITY ASSESSMENT OF OMNIDIRECTIONAL IMAGES
The study constructs an equirectangular OIQA image set with controlled distortions and conducts VR quality ratings while recording head and eye movements. Its protocol combines standardized single-stimulus scoring with calibrated viewing and randomized presentation.
- Database construction: The database contains 336 equirectangular images: 16 raw images and 320 distorted images with resolutions from 11332×5666 to 13320×6660.
- Database construction: Source images were professionally captured, closely matched in resolution and perceptual quality, and checked to reduce intrinsic artifacts.
- Database construction: Four distortion types—JPEG, JPEG2000, Gaussian blur, and white Gaussian noise—were applied at five levels each.
- Database construction: The experiment uses overlapping split images to simulate realistic image stitching after camera-array capture.
- Subjective experiment methodology: Single-stimulus testing and 10-point categorical ratings were selected because subjects viewed one image at a time in an HMD.
- Subjective experiment methodology: Twenty subjects viewed 16 raw images for 20 seconds each, completed training, then rated randomized images while eye and head movements were recorded.
C. Data Processing and Analysis
The subjective experiment produces three data streams for subsequent processing: quality scores, head movements, and eye movements.
- The study collects raw subjective quality scores for all images together with head-movement and eye-tracking data.
1) Subjective Quality Score Processing and Analysis:
Subjective quality scores are aggregated into mean opinion scores and cleaned by removing observations identified as outliers. The score distribution spans the perceptual quality range, with many high-quality images receiving 9 or 10 points.
- Mean opinion score for each image is computed from the scores assigned by all subjects.
- The 3σ principle is used to remove scores far from the mean as outliers.
- Subjective scores span the full perceptual quality range, with many images reaching 9 or 10 points.
2) Visual Attention Data Processing and Analysis:
The study converts head and eye movements into saliency maps for omnidirectional images, capturing both viewing direction and gaze. These maps emphasize that visual attention is concentrated in limited, content-relevant regions.
- The processing overlays subjects’ view-direction positions and eye fixations, filters saccades, and projects attention data into equirectangular images.A 3.34° visual-angle Gaussian filter is applied to fixation maps before back-projection.
- Salient regions mainly cluster near the equator because observers view only part of the scene and less often view the equirectangular image’s top and bottom.The OIQA database includes subjective quality scores and saliency data, including attention data from both original images and formal rating experiments.
- Head-only maps encode view direction, whereas head-eye maps encode human eye gaze.The two maps are similar overall but differ in detailed highlighted regions.
A. Experimental Protocol
The evaluation tests nine full-reference IQA models after mapping their predictions to subjective ratings. Performance is measured with correlation and error criteria reported in Table I.
- Nine state-of-the-art full-reference IQA models are evaluated on the OIQA database.The models include FSIM, GMSD, GSI, IW-SSIM, MS-SSIM, PSNR, SSIM, VIF, and VSI.
- A five-parameter logistic function maps each model’s predicted scores to subjective quality ratings before comparison.x denotes predicted scores, f(x) the mapped score, and β_i the fitted parameters.
- Performance is assessed using Pearson linear correlation coefficient, Spearman rank correlation coefficient, and root mean square error.The resulting performance of all nine models is listed in Table I.
B. Performance Comparison
FSIM, GSI, and VSI perform best on the OIQA database, while most other state-of-the-art models experience a performance drop relative to traditional-image IQA.
- FSIM, GSI, and VSI are the three best-performing metrics on the OIQA database.Their performance is described as fairly good, and they also perform well on traditional IQA databases.
- Most other state-of-the-art IQA models perform poorly and undergo a performance drop when transferred from traditional images to omnidirectional images.The authors identify substantial room for improving these models.
C. Differences between Omnidirectional IQA and Traditional IQA
The OIQA results reveal a distortion-specific mismatch between objective predictions and subjective ratings. White Gaussian noise receives unusually high subjective scores, unlike the more consistent behavior often seen in traditional IQA.
- White Gaussian noise scatter points remain far from fitted curves and are generally higher than points for other distortion types across representative IQA models.This pattern appears for FSIMc, VSI, MS-SSIM, and PSNR, and is also observed across various other IQA models.
- The models predict lower quality than the ideal values for white Gaussian noise, and the authors attribute the mismatch to subjective ratings rather than objective models.The paper contrasts this with the more consistent predictions typically observed across distortions in traditional images.
- The authors suggest that VR viewers may prefer high-frequency content because it provides a more comfortable visual experience under limited HMD display effects.Gaussian noise adds high-frequency information, whereas the other three distortions reduce high-frequency information and image details.
IV. CONCLUSION AND FUTURE WORK
The paper builds and releases the OIQA database with subjective quality, view-direction, and eye-tracking data, then evaluates objective IQA models in VR. Results indicate that VR observers favor high-frequency content and image details, while visual saliency may improve omnidirectional IQA.
- The released database includes subjective quality scores, view-direction information, and eye-tracking data collected during VR experiments.
- Nine objective IQA models are compared on OIQA to examine omnidirectional image-quality assessment in VR.
- Humans prefer high-frequency content and image details in VR HMDs, and losing details can substantially harm visual experience.
- Visual saliency may promote omnidirectional IQA performance because visual attention is extremely important in the VR environment.