PaperlayerView exact revision on arXiv

Source-linked AI summary

GC360IQ: Generic-to-Individualized Quality Assessment for Stitched 360-Degree Panoramas

Jinghan Zhou, Zhou Wang

arXiv:2609.08118v1eess.IV

TL;DR

MOS-based IQA does not capture how individual observers consistently differ in judging stitched-panorama artifacts. GC360IQ combines artifact-aware generic prediction with a preference embedding adapted from anchor ratings, and reports improved individual prediction alongside structured observer differences.

  • Problem

    MOS-based IQA can conceal consistent differences in observers’ scoring tendencies and sensitivity to luminance inconsistency and detail loss in stitched panoramas.

    Source: §I · p. 1

  • Method

    GC360IQ builds a database with individual ratings and uses unblended views for generic quality modeling, then applies MAP adaptation in a learned preference space using anchor ratings.

    Source: §I · pp. 1–2, §V · p. 11

  • Results

    The generic model outperforms representative IQA methods, and subject adaptation improves individual quality prediction; observer residuals show structured variation related to scoring tendency and artifact sensitivity.

    Source: §I · p. 2, §V · p. 11

  • Takeaways & Limitations

    The results support modeling individual ratings as a prediction target rather than treating observer differences only as variation around MOS.

    Source: §I · p. 2, §V · p. 11

Abstract

from arXiv · show

Existing image quality assessment (IQA) methods typically predict mean opinion scores (MOSs) but struggle to capture variations in individual subject behaviors. This limitation is highly pronounced in immersive visual applications such as stitched 360-degree panoramas, where user opinions diverge drastically based on personal sensitivity to blending-induced luminance inconsistency, detail loss, and geometric misalignment. Here we propose GC360IQ, a novel Generic-to-Individualized IQA framework that establishes a learned explicit feature space to characterize individual subject behaviors. First, we construct a specialized 360-degree panorama database focusing on blending-induced luminance and detail degradation while minimizing geometric misalignment, providing multidimensional quality ratings alongside complete individual scores. Second, we develop a generic quality model that utilizes unblended views as a perceptual reference. Dual feature extraction branches capture gradient and structural information specifically around stitching regions to predict baseline quality. Third, we construct a compact preference embedding space that acts as an explicit feature domain to model each subject's deviation from generic quality perceptions. We also introduce a maximum a posteriori (MAP) adaptation mechanism. By leveraging a preference prior learned within our explicit feature space, a new subject's unique behavioral embedding is mapped progressively with an increasing number of anchor ratings. Experiments demonstrate that the generic model provides accurate stitching quality predictions and that subject adaptation further improves individual score predictions. Deeper analysis of the collected ratings and learned embeddings reveals that observer differences contain structured variation related to scoring tendencies and sensitivity to stitching artifacts, rather than merely random rating noise.

I. INTRODUCTION

MOS can conceal stable differences in how observers judge the same image, while stitched panoramas add localized blending artifacts that individuals may weight differently. GC360IQ addresses this with a subject-level database and a framework for generic and individualized prediction; existing panorama databases offer limited isolation of blending-related luminance and detail artifacts from geometric artifacts.

I. INTRODUCTION

  • Individual observers may differ in their scoring tendencies and sensitivity to localized luminance inconsistency and detail loss, differences that MOS can conceal.The paper frames these differences as potentially systematic rather than only random rating noise.Source: §I · p. 1
  • GC360IQ pairs artifact-aware generic quality modeling with a compact preference space and MAP adaptation from limited anchor ratings.Its database focuses on blending-induced luminance inconsistency and detail loss while minimizing geometric misalignment, with multidimensional annotations and complete individual scores.Source: §I · pp. 1–2

A. 360-Degree Image Quality Databases

  • Existing 360-degree quality databases cover homogeneous and spatially varying distortions, but stitched-image databases provide limited support for isolating blending-induced luminance inconsistency and detail loss from geometric artifacts.The cited databases include panoramas with artifacts from alignment and panorama generation, or outputs from different stitching algorithms.Source: A. 360-Degree Image Quality Databases · p. 2

B. 360-Degree Image Quality Assessment

Prior 360-degree IQA methods address spherical sampling, viewport quality, and spatially varying distortions, while stitched-image methods assess aggregate quality. Individualized assessment treats observer variation as a prediction target, but individualized prediction for blending artifacts in stitched panoramas has received little attention.

B. 360-Degree Image Quality Assessment

  • 360-degree IQA methods adapt conventional metrics to spherical geometry and use viewport, saliency, attention, or viewing-behavior models for quality assessment.Examples include WS-PSNR and WS-SSIM, VGCN, ST360IQ, Assessor360, Max360IQ, and IQCaption360.Source: B. 360-Degree Image Quality Assessment · p. 2
  • Stitched-image methods study stitching distortions and artifacts, but existing methods remain centered on aggregate quality rather than individual responses to blending artifacts.The cited factors include geometric misalignment, ghosting, luminance inconsistency, and structural degradation.Source: B. 360-Degree Image Quality Assessment · p. 2

C. Subject Variation and Individualized Visual Assessment

  • MOS and observer-bias models use individual ratings to improve aggregate estimates, whereas individualized assessment treats observer variation as part of the prediction target.Prior individualized work includes subject attributes in aesthetic assessment and user characteristics in quality-of-experience models.Source: C. Subject Variation and… · pp. 2–3
  • Individualized quality prediction for blending artifacts in stitched 360-degree panoramas has received little attention.The framework overview depicts generic modeling of luminance inconsistency and detail loss followed by adaptation to each subject using limited ratings.Source: C. Subject Variation and… · p. 3, Figure 1 · p. 3

III. PROPOSED METHOD

GC360IQ extracts seam-region views from stitched panoramas and their unblended inputs, then uses separate artifact branches to predict generic quality. It fuses branch features for an overall score and reuses intermediate features for individualized prediction.

III. PROPOSED METHOD

  • Local viewports from stitched panoramas and unblended inputs feed separate luminance-inconsistency and detail-loss branches; their features are fused for overall quality and support subject adaptation.A learned preference prior and MAP adaptation use a new subject’s anchor ratings to estimate an embedding that adjusts the generic prediction.Source: §III · p. 3

A. Preprocessing

  • Each sample pairs stitched and unblended patch stacks with a binary seam mask that retains seam regions and reduces non-overlapping areas’ influence.The preprocessing projects each panorama into seven viewport directions and crops perspective views resized to 256 × 256 pixels.Source: A. Preprocessing · p. 3

B. Generic Quality Model

  • The generic model uses MOS as its training target and predicts overall quality by fusing features from separate luminance-inconsistency and detail-loss branches.The model also provides image representations for subject adaptation.Source: B. Generic Quality Model · pp. 3–4
  • The luminance branch estimates inconsistency from stitched-versus-unblended spatial gradients, while the detail-loss branch uses luminance, contrast, and SSIM maps.Intermediate features from both branches are concatenated into a 128-dimensional representation for the overall-quality network.Source: B. Generic Quality Model · pp. 3–4

C. Individualized Quality Model

The individualized model represents each subject’s score as a generic prediction plus a residual modeled with a subject-specific preference embedding. A prior learned from training subjects regularizes MAP adaptation from a new subject’s anchor ratings.

C. Individualized Quality Model

  • Each subject’s quality score is modeled as a generic prediction plus a residual parameterized by a preference embedding.The residual is the subject’s deviation from the generic prediction.Source: C. Individualized Quality Model · p. 4
  • The image representation combines branch-specific artifact features and overall-quality features, then PCA reduces it to 64 dimensions before adding a bias dimension.The PCA projection is estimated using training images and remains fixed during testing.Source: C. Individualized Quality Model · p. 4
  • Training-subject preference embeddings are estimated by ridge regression from image representations and residual scores, with the bias term left unregularized.The residual is each subject’s score minus the generic prediction.Source: C. Individualized Quality Model · p. 4
  • A multivariate Gaussian prior is estimated from training-subject embeddings, with covariance shrinkage toward a diagonal matrix and a small stability constant.The prior remains fixed when adapting to a new subject.Source: C. Individualized Quality Model · p. 4
  • MAP adaptation fits a new subject’s anchor residuals while regularizing the embedding toward the learned prior; without ratings, the estimate is the prior mean.The adapted embedding predicts scores for previously unrated images.Source: C. Individualized Quality Model · p. 4

IV. EXPERIMENTS AND RESULTS

The experiments evaluate database construction and analysis, the generic quality model, and subject-level prediction. The database construction process captures varied scenes, minimizes geometric misalignment, and applies different blending algorithms.

IV. EXPERIMENTS AND RESULTS

  • The experiments cover database construction and rating analysis, generic-model pretraining, comparison and ablation, and subject-level prediction with preference-space analysis.The database subsection describes image acquisition, panorama generation, and subjective assessment.Source: §IV · p. 5, A. Database Construction and Analysis · p. 5

A. Database Construction and Analysis

  • A seven-camera rig captures six equatorial views and one zenith view, with automatic exposure and simultaneous triggering.The simultaneous captures provide overlap for stitching.Source: A. Database Construction and Analysis · p. 5
  • The 20 source scenes comprise 10 outdoor daytime, 4 nighttime, and 6 indoor scenes spanning varied illumination, structural complexity, and visual content.The scenes provide source material for stitched panoramas and quality assessment.Source: A. Database Construction and Analysis · p. 5
  • The captured views are aligned to minimize geometric distortions before blending algorithms are applied, isolating blending artifacts from geometric misalignment.The panoramas are processed using PTGui Pro and a panoramic alignment method.Source: A. Database Construction and Analysis · p. 5

1) Image Acquisition and Stitching:

The database combines five blending algorithms with controlled subjective ratings of stitched panoramas. Examples show that subjects can disagree on images with mid-range MOS, motivating analysis of subject-dependent preferences.

1) Image Acquisition and Stitching:

  • Five blending algorithms applied to 20 scenarios produce 100 stitched panoramas with luminance inconsistency and detail loss as primary blending artifacts.The algorithms are Naive Blending, FB, MBB, MVCB, and MPB.Source: 1) Image Acquisition and Stitching · p. 5

2) Subjective Test:

  • Thirty subjects rated 100 panoramas on luminance inconsistency, detail loss, and overall quality using a five-point scale after viewing unblended-input and seam visualizations.The HMD test let observers explore panoramas in 360 degrees, and image presentation order was randomized.Source: 2) Subjective Test · pp. 5–6

3) Subject Disagreement in Quality Assessment:

  • For image 75, MOS is 3.00 while 9 subjects rate it 2 and 9 rate it 4; for image 53, MOS is 3.17 despite ratings spanning 1 to 5.The examples associate disagreement with luminance inconsistency and texture or detail loss.Source: Figure 4 · p. 6, 3) Subject Disagreement in Quality… · p. 6
  • The split ratings suggest that subjects differ in sensitivity to local stitching artifacts, motivating analysis of whether deviations from average ratings form a structured preference space.The examples include luminance inconsistency and severe texture or detail loss.Source: 3) Subject Disagreement in Quality… · p. 6

B. Generic Quality Prediction

The generic quality model is evaluated on GC360IQ, with pseudo-data pretraining used to address the database’s limited size. The section describes pseudo-data construction and the model’s training and testing procedure.

1) Pseudo Dataset Construction:

  • Seven virtual views per HDR scene were tone-mapped independently, assigned three exposure-difference levels, and processed with five blending algorithms.The resulting 3 × 5 exposure-and-blending combinations produced 15 panoramas per scene.Source: 1) Pseudo Dataset Construction · p. 7
  • The pseudo dataset contains 3,750 stitched panoramas with rank-order labels for pretraining before fine-tuning on GC360IQ ratings.It was built from 250 HDR panoramas, with 15 stitched results generated per scene.Source: 1) Pseudo Dataset Construction · pp. 6–7

2) Training and Testing:

  • Training and testing used scenario-wise splits, keeping all five blended panoramas from each GC360IQ scenario together.The GC360IQ split used 10 training and 10 test scenarios; pseudo-data HDR content used an 80–20 split.Source: 2) Training and Testing · p. 7
  • After pretraining, the artifact branches were fine-tuned on GC360IQ, and their features were fused to train the overall-quality predictor.Fine-tuning first optimized the final layer, then unfroze the network; the overall-quality network used the same two-stage strategy.Source: 2) Training and Testing · p. 7

3) Performance Comparison:

GC360IQ achieves the strongest reported generic-quality comparison on the GC360IQ database. The ablation results also show benefits from pseudo-data pretraining after fine-tuning.

3) Performance Comparison:

  • GC360IQ achieves the highest PLCC and SRCC and the lowest RMSE among compared methods on the GC360IQ database.The authors report that these results show the best agreement with subjective overall-quality scores.Source: 3) Performance Comparison · p. 7

4) Pseudo-data Pretraining Ablation:

  • PLCC and SRCC indicate that useful stitching-related representations are learned before fine-tuning, while high direct-pretraining RMSE is expected from rank-order labels.The pseudo labels provide rank-order supervision rather than absolute MOS values.Source: 4) Pseudo-data Pretraining Ablation · p. 8
  • Fine-tuned pretrained models outperform scratch-trained models for luminance inconsistency, detail loss, and overall-quality prediction.The authors interpret this as evidence that pseudo-data pretraining learns artifact-aware quality representations.Source: 4) Pseudo-data Pretraining Ablation · p. 8

C. Subject-Adaptive Quality Prediction

Subject adaptation improves individual quality prediction beyond the generic estimate, including when predicting a new subject from a limited number of anchor ratings. Gains vary across subjects, with larger improvements for those less well represented by the average profile.

1) Subject-Level Prediction Performance:

  • On held-out images, subject adaptation improves PLCC and SRCC and reduces RMSE compared with generic prediction.The evaluation averages results over five repeated splits.Source: 1) Subject-Level Prediction Performance · p. 8
  • In the new-subject protocol, 20 images form a progressively ordered anchor pool and 80 fixed images form the test set.The generic estimate is the leave-one-subject-out MOS; with K = 0, the embedding is the prior mean and produces the generic baseline.Source: 1) Subject-Level Prediction Performance · p. 8
  • Across 30 subjects and 15 random anchor orders, PLCC consistently increases as more anchor ratings are provided.The reported experiment treats each existing subject as a new subject in turn.Source: 1) Subject-Level Prediction Performance · pp. 8–9
  • Adaptation gains vary by subject: those with larger baseline errors from the average profile often improve substantially, while well-approximated subjects gain less.Fig. 6 reports RMSE trajectories separately for all 30 subjects.Source: Figure 6 · p. 8, 1) Subject-Level Prediction Performance · p. 9

2) Subject Preference Space:

Subject-level deviations form a structured preference space rather than random rating noise. Its dimensions relate to observers’ overall scoring tendencies and sensitivities to luminance inconsistency and detail loss.

2) Subject Preference Space:

  • The residual matrix is formed by subtracting MOS from each subject’s overall quality score, representing deviation from the image’s average opinion.The residual is defined for each image and subject.Source: 2) Subject Preference Space · p. 9
  • The leading singular values of the original residual matrix exceed the shuffled baseline’s permutation mean, indicating structure beyond random scoring fluctuations.The shuffle removes subject-specific patterns while preserving the residual distribution.Source: 2) Subject Preference Space · p. 9
  • Each subject’s K = 20 embedding is projected onto the first four principal components for analysis of subject-level artifact sensitivity.K = 20 uses all anchor images for adaptation.Source: 2) Subject Preference Space · p. 9
  • Subject residuals are fitted using overall scoring tendency and luminance-inconsistency and detail-loss artifact strengths.The fitted coefficients measure each subject’s scoring tendency relative to MOS and sensitivity to each artifact type.Source: 2) Subject Preference Space · p. 9
  • The first preference dimension correlates strongly with overall scoring tendency, while the second and third relate more to luminance and detail-loss sensitivities.The four preference dimensions are compared with independently fitted perceptual factors.Source: 2) Subject Preference Space · p. 9, Figure 8 · p. 9

3) Preference Deviation and Adaptation Gain:

GC360IQ’s generic model outperforms representative IQA methods, and adaptation improves individual quality prediction. Adaptation gains vary with preference deviation, while the conclusion links observer differences to structured scoring tendencies and artifact sensitivities.

3) Preference Deviation and Adaptation Gain:

V. CONCLUSION

  • GC360IQ combines a 100-panorama database with multidimensional ratings and 30 subjects’ individual scores, a reference-based generic model, and MAP adaptation into a preference space.The generic model uses unblended views as a reference and extracts gradient and structural features around stitching regions.Source: §V · p. 11
  • The generic model outperforms representative IQA methods, and subject adaptation improves individual quality prediction.The paper concludes that individual ratings are useful prediction targets rather than variation to treat only around MOS.Source: §V · p. 11
  • The conclusion characterizes observer differences as structured variation associated with scoring tendency and sensitivity to luminance inconsistency and detail loss.Subjects farther from the shared profile also obtain larger adaptation gains.Source: §V · p. 11
Loading 2609.08118v1…