Pano: Optimizing 360° Video Streaming with a Better Understanding of Quality Perception

Introduction

• At the same perceived quality, streaming a panoramic sphere at 40 pixels per degree requires 400 Mbps versus roughly 5 Mbps for a desktop full-HD video.The paper says viewport-driven approaches [32] still require at least twice screen-size bandwidth, risk prediction-error-induced rebuffering or quality drops, and can incur tile overhead.Source: §1 · p. 1
• Pano’s quality model captures sensitivity changes tied to relative viewpoint and object motion, depth-of-field differences, and recent viewport luminance changes.The paper assumes head-mounted-device movement approximates viewpoint movement and that the object closest to the viewpoint is being watched.Source: §1 · pp. 1–2
• Pano’s variable-sized tiling groups regions so users tend to have similar sensitivity within each tile, balancing perceptual fit against encoding overhead.Uniform tiles may be too coarse to reflect sensitivity variation or too fine-grained to contain encoding overhead.Source: §1 · p. 2
• Pano adapts quality using predicted viewpoint movements and sensitivity estimates, with offline precomputation and online matching to support deployment in existing delivery infrastructure.It estimates sensitivity by predicting ranges of viewpoint speed, luminance, and depth-of-field, rather than requiring exact viewpoint predictions.Source: §1 · p. 2
• Pano increased mean opinion score (MOS) by 25–142% without more bandwidth, saved up to 46% bandwidth, or reduced buffering by 60–98% without perceived-quality loss.Evaluation combined user studies with 20 participants and 7 videos and trace-driven simulations with 48 users and 18 videos.Source: §1 · p. 2

2.1 Background of 360° video streaming

• Viewport-driven streaming fetches only the predicted viewport, risking user experience when prediction is wrong, while fine-grained tiles or viewpoint-specific encodings can significantly increase video size.Source: §2.1 · p. 3

2.2 New quality-determining factors

• Prior work assumes 360° viewers perceive viewport quality like viewers of non-360° video, limiting improvement and requiring more bandwidth for the larger viewport.Source: §2.2 · p. 3
• Viewpoint movement changes distortion sensitivity: faster motion reduces sensitivity to background distortion, luminance transitions briefly reduce sensitivity, and regions with DoF closer to the viewpoint are more sensitive.These effects arise from viewpoint movements, unlike prior perceptual approaches based on video content; DoF-based savings come from lowering quality for objects whose DoF differs greatly from the viewpoint.Source: §2.2 · p. 3

2.3 Potential gains

• The three viewpoint-related factors exceed user-study-derived sensitivity thresholds for 5–40% of the time, indicating opportunities to tolerate more distortion.The thresholds are 10 deg/s for viewpoint speed, a 200 gray-level luminance change, and a 0.7-diopter DoF difference; users can tolerate 50% more distortion when a threshold is exceeded.Source: §2.3 · pp. 3–4, Figure 3 · p. 4

• Pano models quality sensitivity as a function of viewpoint speed, luminance change, and depth-of-field difference, decoupling movement effects from content and exploiting largely independent factor impacts.This avoids profiling every possible combination of factor values and video content.Source: §3 · p. 4
• Almost 200% more video size from naive 12×24 tiling than a coarse 3×6 grid motivates Pano’s handful of variable-size tiles, which preserve coarse granularity while assigning higher quality to more-sensitive regions.Tiles are designed to group regions where users have similar sensitivities to distortion, based on historical viewpoint trajectories.Source: §3 · p. 4
• Pano adapts tile quality using estimated viewpoint-speed ranges rather than precise movements, tolerating prediction errors and bandwidth variation while conservative choices still outperform baselines that ignore viewpoint movement.Recent history can provide a reliable lower bound on speed during rapid movement, though underestimation may lead to higher-than-necessary quality.Source: §3 · p. 4
• Pano stores a quality-level lookup table in the video manifest so clients can estimate perceived quality without video-content access, requiring only minor provider and client changes and none to CDNs or HTTP streaming.The design targets compatibility with the existing client-driven streaming protocol.Source: §3 · p. 4

4.1 A general video quality framework

• Pano incorporates the 360°-specific factors into PSPNR, a perceived-quality metric that extends PSNR by filtering distortions below the Just-Noticeable Difference.The model raises JND as relative viewpoint speed, DoF difference, or luminance change increases.Source: §4.1 · pp. 4–5

4.2 Profiling JND of 360° videos

• JND rose monotonically with viewpoint speed, luminance change, and sharper DoF differences; tested pairs of factors appeared independent, while current luminance’s impact was non-monotonic.These effects were assessed in a user study with 20 participants watching 43 short test videos.Source: §4.2 · p. 5
• 360JND-based PSPNR predicted MOS much more accurately than traditional PSPNR and PSNR, indicating that the three 360°-specific factors strongly influence perceived quality.The comparison used ratings from 20 participants across 21 360° videos.Source: Figure 8 · p. 6, §4.2 · p. 6
• 360JND multiplies content-dependent JND by viewpoint-speed, luminance-change, and DoF-difference multipliers.The model separates content-dependent JND, which can be precalculated, from the action-dependent ratio, which must be determined in real time.Source: §4.2 · p. 6

5 PANO: VIDEO TILING

• Pano splits each equal-duration chunk into a 12×24 grid of square unit tiles, scores each by quality growth across encoding levels, and groups them into variable-size tiles, defaulting to 30.The grouped tiles are used to encode the video.Source: §5 · p. 6
• Grouping minimizes area-weighted variance in unit-tile efficiency scores, so similarly scored tiles can receive similar playback quality while fewer tiles reduce video size.Fewer tiles also avoid re-encoding boundaries between small tiles.Source: §5 · p. 6
• Pano’s top-down partitioning repeatedly splits rectangles until it has N coarse tiles; offline PSPNR averages across historical viewpoint trajectories inform scores, and fixed tiles avoid playback re-encoding.Figure 10 indicates that a reliably estimated lower bound on viewpoint-moving speed is often sufficient for accurate PSPNR estimation.Source: §5 · p. 6, Figure 10 · p. 7
• At most 10% worse than mean quality was the lowest perceived quality across users in the traces, despite possible suboptimal quality for atypical viewing behavior.The authors identify averaging PSPNR over historical viewpoint movements as a potential source of this limitation.Source: §5 · p. 7

6.1 Robust quality adaptation

• Pano uses MPC to set each chunk’s bitrate, then chooses tile quality levels to maximize PSPNR while keeping total tile size within that bitrate.It prunes tile assignments when another assignment is strictly better in both PSPNR and total size.Source: §6.1 · p. 7
• Approximate ranges for viewpoint speed, DoF difference, and luminance change can guide quality selection; lower bounds from recent history enable conservative decisions under arbitrary motion.Pano also prunes the search space by excluding assignments that are strictly worse in both PSPNR and tile size.Source: §6.1 · p. 7

6.2 DASH-compatible design

• Pano’s PSPNR adaptation conflicts with DASH’s passive-server design, so it precomputes PSPNR for representative viewpoint conditions and sends a lookup table in the manifest for client-side estimation.The table covers n representative values for each of viewpoint speed, DoF difference, and luminance change, yielding n^3 combinations.Source: §6.2 · p. 7, Figure 11 · p. 7

6.3 System optimization

• Pano compresses a 5-minute video’s PSPNR manifest lookup table from 10 MB to about 50 KB by indexing with combined action-dependent ratios and representing each tile’s relationship with a power function.The power-function representation requires only two parameters per tile.Source: §6.3 · pp. 7–8
• Sampling one frame in ten and reusing its PSPNR for the other nine reduces PSPNR computation overhead by 90% while performing as effectively as per-frame calculation.Original per-frame PSPNR calculation can be about 50% slower than encoding the same video.Source: §6.3 · p. 8

7 IMPLEMENTATION

• Pano’s 15K-line prototype preprocesses video into 1-second chunks, 30 default tiles at five QP levels, extracting object trajectories, luminance, and DoF.Yolo[54] detects objects in each second’s first frame, and tracking logic [38] follows them through the remaining frames. The manifest records tile coordinates, average luminance and DoF, object trajectories, and PSPNR lookup tables.Source: §7 · p. 8
• Pano leaves the DASH video server unchanged, requiring changes only to the client-side player.This follows recent work on 360° video.Source: §7 · p. 8
• The client predicts viewpoints 1–3 seconds ahead, estimates tile PSPNR from viewpoint speed, luminance change, and DoF difference, then allocates tile quality levels within the chunk bitrate.The client parses the augmented manifest and uses the PSPNR lookup table before applying Pano’s tile-level bitrate allocation logic.Source: §7 · p. 8
• The client fetches chunk tiles as separate HTTP objects, decodes them in parallel, and stitches them into panoramic frames with 1 ms latency.Manifest tile coordinates determine placement; row-major copying with compiler-optimized memcpy can make stitching efficient.Source: §7 · p. 8

8.1 Methodology

• The evaluation uses a 20-participant survey and trace-driven simulations, comparing Pano with Flare and ClusTile using shared viewpoint-prediction and chunk-level bitrate-adaptation logic.Participants rated seven videos across two methods and two bandwidth conditions; the study also evaluates PSPNR and buffering ratio.Source: §8.1 · p. 9

8.2 End-to-end quality improvement

• Pano raises user ratings by 25–142% at nearly the same bandwidth as Flare, and across simulations achieves higher PSPNR, lower buffering, or both.The simulations cover 18 videos across four genres and two network traces; gains vary with viewpoint dynamics, with more dynamic movement creating more opportunity to reduce quality without hurting perceived quality.Source: §8.2 · pp. 9–10, Figure 13 · p. 9, Figure 15 · p. 10

8.3 Robustness

• With 40-degree viewpoint noise, median PSPNR prediction deviates by 7 dB, and Pano continues to outperform Flare as viewpoint noise rises, with diminishing gains.As throughput prediction error increases, Pano's quality degrades similarly to Flare's.Source: §8.3 · p. 10, Figure 16 · p. 11

8.4 System overhead

• Pano reduces client processing overhead and startup delay versus Flare, while preprocessing time remains on par despite added PSPNR computation and variable-size tiling.The larger manifest adds startup overhead, but this is offset by reduced first-chunk loading time.Source: §8.4 · pp. 10–11, Figure 17 · p. 11

8.5 Bandwidth savings

• At PSPNR 72 (approximately MOS 5), JND awareness saves 17% bandwidth, adding 360JND saves a further 11%, and variable-size tiling saves another 17%.Each saving is measured against the preceding approach in the breakdown over the viewport-driven baseline.Source: §8.5 · p. 11, Figure 18 · p. 11
• Pano achieves the same PSPNR with 41–46% less bandwidth than the viewport-driven baseline on real throughput traces.The comparison uses real throughput traces; Figure 18(b) presents the bandwidth-savings result.Source: Figure 18 · p. 11, §8.5 · p. 12

• The 360JND model was built on a survey in which participants watched and rated videos spanning viewpoint speeds, depth-of-field differences, and luminance changes, following related-work methodology.Source: §9 · p. 12
• The study varied 360° video-specific factors in prescribed ways that may not match real-world viewing; simulated viewpoint motion was horizontal and constant-speed.Source: §9 · p. 12
• 360JND was tested with only two factors non-zero and assumes independent JND effects that can be multiplied; Figure 8 suggests finer profiling could help.Source: §9 · p. 12

Viewport tracking

• Pano predicts viewpoint location and additional quality factors—viewpoint speed, luminance, and depth of field—building on prior viewport-prediction approaches.Prior methods extrapolate viewport from movement history, cross-user similarity, or content analysis; some instead enlarge the recent viewport, which can still miss rapid movement.Source: §10 · p. 12

360° video tiling

• Pano uses variable-size tiles aligned with the spatial distributions of its quality-determining factors, unlike common grid tiling and ClusTile’s clustered small tiles.Increasing tile count adds encoding overhead, while tile-based encoding enables viewport-adaptive streaming [32].Source: §10 · p. 12

Bitrate adaptation in 360° videos

• Unlike viewport-distance-based tile allocation alone, Pano also weights each tile by users’ sensitivity to its quality distortion.360° video adaptation spatially allocates bitrate among tiles, whereas non-360° video adaptation changes bitrate at chunk boundaries [58].Source: §10 · p. 12

Just-Noticeable Distortion and perceived quality

• Pano applies JND-based perceived-quality modeling to 360° video, accounting for interactive behaviors such as viewpoint movement to improve quality with less bandwidth.Prior JND studies measure human visual sensitivity, but most existing work targets video coding and non-360° video.Source: §10 · p. 12

11 CONCLUSION

• 25%-142% higher mean opinion score with the same bandwidth consumption is reported for Pano over the state-of-the-art.The conclusion describes this as an experimental result.Source: §11 · p. 12
• Users perceive 360° video quality differently from non-360° video quality, challenging prior solutions’ traditional-video assumptions.Those assumptions limited opportunities to improve 360° video through bandwidth-quality tradeoffs.Source: §11 · p. 12
• Pano revisits perceived quality metrics, video encoding schemes, and quality adaptation logic based on those differences in perception.The paper presents Pano as a concrete design inspired by these ideas.Source: §11 · p. 12

• The appendix presents the detailed methodology for modeling 360° video JND.Source: A APPENDIX · p. 14

A.1 Survey process

• The study used 20 participants aged 20–26, who also completed the §8 performance evaluation, and tested videos through an Oculus headset.The paper notes that the limited participant pool could affect results; headset parameters are summarized in Table 4.Source: §A.1 · p. 14
• 20 participants’ average first-perceived distortion level defined each video’s JND across 43 artificially generated videos shown in random order.Participants reported whether they perceived distortion each time its level increased.Source: §A.1 · p. 14

A.2 Test videos

• Test videos placed a 64×64-pixel, grey-level-50 square at screen center, then distorted half its pixels by varying Δ from 1 to 205 within the 0–255 range.This setup controlled noise on the foreground object to emulate increasing quality distortion.Source: §A.2 · p. 14
• To test relative viewpoint speed, participants looked at a fixed central red spot while the distorted object moved horizontally at speeds from 0 to 20 deg/s.They reported whether they perceived distortion on the moving object.Source: §A.2 · p. 14
• For luminance-change tests, background luminance fell from д + l to д after 5 seconds; with д fixed at 0, l ranged from 0 to 240 grey levels.Participants reported distortion after the drop and always did so within 3 seconds, which the authors treated as a reasonable JND indicator under that change.Source: §A.2 · p. 14
• For depth-of-field tests, participants focused on a static spot at a DoF difference of 0, 0.67, 1.33, or 2 dioptres from the distorted foreground object.They reported when they first perceived the object’s distortion.Source: §A.2 · p. 14
• The tests also varied viewpoint speed jointly with DoF difference and jointly with object luminance change, with results shown in Figure 7.Each factor had otherwise been tested separately while the others were held at zero.Source: §A.2 · p. 14