Source-linked AI summary
Comyco: Quality-Aware Adaptive Video Streaming via Imitation Learning
Tianchi Huang, Chao Zhou, Rui-Xiao Zhang, Chenglei Wu, Xin Yao, Lifeng Sun
TL;DR
Learning-based ABR faces sample inefficiency and limited attention to perceptual video quality. Comyco trains from expert trajectories and targets higher-quality chunks. The paper reports improved training efficiency, QoE, and video quality relative to prior methods.
Problem
Learning-based ABR faces inefficient training from collected experience and QoE metrics that overlook perceptual video quality.
Source: §1 · p. 1, §2.2 · p. 2Method
Comyco uses imitation learning to train from expert policies and builds a quality-aware ABR system that selects chunks using perceptual video quality.
Source: §1 · p. 1Results
Comyco improves average QoE by 7.5%–16.79% over prior methods and average video quality by 7.37% over Pensieve at the same rebuffering time.
Source: §1 · p. 2Takeaways & Limitations
Trace-driven emulation and real-world deployment show improved performance and faster training for Comyco.
Source: §8 · p. 8
Abstract
from arXiv · showhide
Learning-based Adaptive Bit Rate~(ABR) method, aiming to learn outstanding strategies without any presumptions, has become one of the research hotspots for adaptive streaming. However, it typically suffers from several issues, i.e., low sample efficiency and lack of awareness of the video quality information. In this paper, we propose Comyco, a video quality-aware ABR approach that enormously improves the learning-based methods by tackling the above issues. Comyco trains the policy via imitating expert trajectories given by the instant solver, which can not only avoid redundant exploration but also make better use of the collected samples. Meanwhile, Comyco attempts to pick the chunk with higher perceptual video qualities rather than video bitrates. To achieve this, we construct Comyco's neural network architecture, video datasets and QoE metrics with video quality features. Using trace-driven and real-world experiments, we demonstrate significant improvements of Comyco's sample efficiency in comparison to prior work, with 1700x improvements in terms of the number of samples required and 16x improvements on training time required. Moreover, results illustrate that Comyco outperforms previously proposed methods, with the improvements on average QoE of 7.5% - 16.79%. Especially, Comyco also surpasses state-of-the-art approach Pensieve by 7.37% on average video quality under the same rebuffering time.
1 INTRODUCTION
Comyco addresses learning-based ABR’s sample inefficiency and limited use of perceptual video quality by imitating expert policies and selecting for quality. The paper reports faster training and better QoE and video quality than prior methods.
1 INTRODUCTION
- Comyco targets chunks with higher perceptual video quality rather than higher bitrate, using video content, network status, and playback state in its neural network.It uses VMAF to measure quality and proposes a quality-based QoE metric.Source: §1 · pp. 1–2
- The authors introduce a DASH video dataset spanning movies, sports, TV shows, games, news, and music videos.The proposed quality-based QoE metric achieves state-of-the-art performance on Waterloo Streaming SQoE-III.Source: §1 · p. 2
- Comyco improves average QoE by 7.5%–16.79% over prior methods and average video quality by 7.37% over Pensieve at the same rebuffering time.Source: §1 · p. 2
- Comyco requires 1700x fewer samples than recent work by training its policy from expert trajectories supplied by an instant solver.The solver estimates expert actions using a faithful virtual player, and an experience replay buffer stores expert policies for training.Source: §1 · pp. 1–2
- The paper presents imitation learning for ABR training and claims it can accelerate training while improving performance.The authors describe this as, to their knowledge, the first use of imitation learning to accelerate ABR training.Source: §1 · p. 2
2 BACKGROUND AND CHALLENGES
ABR selects video chunks and their quality using network and playback conditions, aiming to deliver high-quality video while avoiding rebuffering. The section identifies challenges in learning-based ABR: inefficient RL training and quality measures that overlook perceptual video quality.
2.1 ABR Overview
- ABR selects the next chunk and its quality using throughput estimates and buffer utilization, with the goal of high quality and avoiding rebuffering.Source: §2.1 · p. 2
2.2 Challenges for learning-based ABRs
- Traditional ABR algorithms [23, 44, 51] use time-series prediction or automation control to decide the next chunk, but their assumptions make consistent performance difficult across considered network scenarios.Source: §2.2 · p. 2
- RL-based ABR training depends on the amount and quality of collected experience; non-optimal samples can prolong convergence to a suboptimal policy [29] [35].Source: §2.2 · p. 2
- Bitrate-based QoE metrics omit perceptual presentation quality; Figure 1 compares bitrate-aware and quality-aware ABR across selected bitrate, buffer occupancy, and VMAF.The comparison uses the same video on Norway network traces.Source: §2.2 · p. 2, Figure 1 · p. 2
3 METHODS
Comyco combines imitation learning with a video quality-aware ABR system. The methods section presents expert-guided policy learning and a system setup that incorporates video, network, and playback information alongside quality-based evaluation and data.
3 METHODS
- Comyco’s methods have two main parts: training the neural network via imitation learning and constructing a complete video quality-based ABR system.Source: §3 · p. 3
3.1 Training ABRs via Imitation Learning
- Imitation learning trains on expert policies while letting the agent explore; unlike ordinary supervised learning, it aims to keep exploration near the expert trajectory and avoid compounding errors.The paper describes supervised learning’s drift into unseen states after a minor action error, which can lead to continuous rebuffering.Source: §3.1 · p. 3
3.2 Video Quality-aware ABR System Setup
- The video quality-aware system combines past network, video content, and playback features, and includes a quality-based QoE metric and a varied DASH video dataset.Source: §3.2 · p. 3
4 SYSTEM OVERVIEW
Comyco combines a neural ABR policy with an instant solver and replay-based training. Its design uses network, video-content, and playback information to select bitrates, while expert actions guide learning.
4 SYSTEM OVERVIEW
- Comyco’s system combines a neural network, ABR virtual player, instant solver, and experience replay buffer.The section introduces the modules, training methodology, and multi-agent framework.Source: §4 · p. 3
4.1 NN Architecture Overview
- The policy takes network, video-content, and playback features as input and outputs probabilities over six bitrate choices.The architecture extracts features with convolutional or fully connected layers, combines them with a GRU, and applies softmax to the output.Source: §4.1 · pp. 3–4
4.2 Instant Solver
- The instant solver selects future chunk bitrates by looking N steps ahead in an offline virtual player and optimizing QoE.It updates virtual time and buffer occupancy using chunk download time and measured throughput; the paper sets N = 8.Source: §4.2 · p. 4
4.3 Choice of Loss Functions for Comyco
- Comyco’s loss combines cross-entropy against the expert action with a policy-entropy term that controls exploration.The entropy term encourages exploration early in training and discourages it later; α controls its strength.Source: §4.3 · p. 4
4.4 Training Comyco with Experience Replay
- An experience replay buffer stores past expert strategies and supplies randomly selected samples during training.The described approach is inspired by off-policy RL methods.Source: §4.4 · pp. 4–5
4.5 Methodology
- Training collects state–expert-action pairs, samples a batch from the buffer, and updates the policy with the Comyco loss.The procedure repeats until convergence.Source: §4.5 · p. 5
4.6 Parallel Training
- The multi-agent framework asynchronously gathers expert actions into a shared replay buffer for central-agent training.Comyco uses 12 agents by default, matching the number of CPU cores on the authors’ PC.Source: §4.6 · p. 5, Figure 5 · p. 5
4.7 Implementation
- The implementation uses TensorFlow and TFlearn for training and network construction, with C++ for the instant solver and virtual player.It uses past sequence length k = 8, seven future chunk features, and Adam with learning rate 10^-4.Source: §4.7 · p. 5
5 QOE METRICS & VIDEO DATASETS
Comyco pairs a video-quality-aware QoE model with a DASH video dataset to support ABR evaluation that accounts for perceptual quality.
5.2 Video Datasets
- The section also introduces a collected DASH video dataset with different VMAF assessments to support video-quality-aware ABR work.The dataset is presented alongside the QoE model as a resource for training and evaluation.Source: §5 · p. 5
5.1 QoE Model Setup
- The QoE model combines perceived video quality, rebuffering time, and positive and negative quality smoothness across chunks.It maps chunk bitrate to perceived quality and weights the terms with parameters α, β, γ, and δ.Source: §5.1 · p. 5
- The proposed QoEv model outperforms recent QoE models on Waterloo Streaming SQoE-III.Table 1 compares QoE models on the SQoE-III dataset.Source: Table 1 · p. 6, §5.1 · p. 6
- VMAF had the highest correlation with MOS among the tested bitrate, SSIM, and VMAF measures, with correlation-coefficient improvements of 16.39%-43.54%.The authors therefore use VMAF as the quality function in their QoE model.Source: §5.1 · p. 6
6 EVALUATION
Comyco is evaluated with trace-driven experiments and real-world deployment against representative ABR baselines. Results report improved training efficiency and QoE across network traces and video types.
6.1 Methodology
- The offline player trains Comyco using network traces and video descriptions; experiments include HSDPA, FCC, Oboe, and synthetic traces.Baselines include throughput-based, buffer-based, MPC, and DRL methods, including Pensieve.Source: §6.1 · p. 6
6.2 Comyco vs. ABR schemes
- 1700x fewer samples and 16x less training time; Comyco raises average QoE 7.5%-17.99% on HSDPA and 4.85%-16.79% on FCC.Figure 7 compares approaches under HSDPA and FCC traces; the prose also reports improved video quality relative to other ABRs.Source: Figure 7 · p. 7, §6.2 · p. 7
- Comyco surpasses Pensieve in 91% of HSDPA sessions and 78% of FCC sessions; its average video quality improves 6.84%-15.64% over other ABRs.The comparison also reports similar performance to state-of-the-art schemes on rebuffering and bitrate changes.Source: §6.2 · p. 7
6.3 Comyco with Multiple Videos
- On Oboe traces, Comyco improves average QoE 2.7%-23.3% over model-based ABRs and 2.8%-13.85% over Pensieve across considered video types.The authors report strong results for movies, news, and sports, but not overwhelming performance for music videos.Source: Figure 9 · p. 7, §6.3 · p. 7
6.4 Ablation Study
- N=9 performs strongly but is inefficient; Comyco uses k=8, while α=0.001 performs best and entropy loss helps; overhead is 229 Kflops, 0.15% of ShuffleNet V2.The authors describe k=8 as balancing performance and cost and consider Comyco deployable on PCs, laptops, or mobile devices.Source: Table 2 · p. 8, Table 3 · p. 8, §6.4 · p. 8
6.5 Comyco in the Real World
- In real-world deployment, Comyco improves average QoE 4.57%-9.93% over Pensieve and 6.43%-9.46% over RobustMPC.The experiments cover 4G/LTE, WiFi, and an international Singapore-to-Beijing link.Source: Figure 10 · p. 8, §6.5 · p. 8
7 RELATED WORK
Prior ABR work includes model-based bitrate-selection heuristics and learning-based approaches, while imitation learning has also been applied in networking.
Model-based
- Model-based ABR methods use throughput predictions, buffer occupancy, or both to select bitrates and manage rebuffering.Examples include FESTIVE, BBA, MPC, and an auto-tuning method.Source: §7 · p. 8
Learning-based
Imitation Learning meets Networking
8 CONCLUSION
The conclusion presents Comyco as a learning-based ABR system that uses imitation learning and video-quality-aware components to improve performance and training speed.
8 CONCLUSION
- Comyco uses imitation learning to guide exploration and exploitation of better ABR policies, addressing sample inefficiency.The conclusion contrasts guided policy learning with stochastic sampling.Source: §8 · p. 8
- The system incorporates video quality into its neural-network architecture, datasets, and QoE metrics.Source: §8 · p. 8
- Trace-driven emulation and real-world deployment show improved performance and faster training for Comyco.Source: §8 · p. 8