PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepQTMT: A Deep Learning Approach for Fast QTMT-based CU Partition of Intra-mode VVC

Tianyi Li, Mai Xu, Runzhi Tang, Ying Chen, Qunliang Xing

arXiv:2006.13125v3eess.IVcs.MM

TL;DR

VVC’s flexible QTMT partitioning greatly increases intra-mode encoding complexity, creating a need for faster CU-partition decisions. The paper learns these decisions with a database and multi-stage early-exit MSE-CNN, achieving large encoding-time reductions with limited BD-BR increases.

  • Problem

    VVC’s QTMT-based CU partition sharply increases intra-mode encoding complexity because partition decisions rely on brute-force RD optimization.

  • Method

    The paper builds a large QTMT-partition database and uses an early-exit multi-stage MSE-CNN, adaptive loss, and multi-threshold decisions to predict CU partitions.

  • Results

    Encoding time decreases by 44.65%∼66.88%, with a 1.322%∼3.188% BD-BR increase on video sequences, outperforming other state-of-the-art approaches.

  • Takeaways & Limitations

    The approach provides a complexity–RD-performance trade-off for accelerating intra-mode VVC encoding.

Abstract

from arXiv · show

Versatile Video Coding (VVC), as the latest standard, significantly improves the coding efficiency over its ancestor standard High Efficiency Video Coding (HEVC), but at the expense of sharply increased complexity. In VVC, the quad-tree plus multi-type tree (QTMT) structure of coding unit (CU) partition accounts for over 97% of the encoding time, due to the brute-force search for recursive rate-distortion (RD) optimization. Instead of the brute-force QTMT search, this paper proposes a deep learning approach to predict the QTMT-based CU partition, for drastically accelerating the encoding process of intra-mode VVC. First, we establish a large-scale database containing sufficient CU partition patterns with diverse video content, which can facilitate the data-driven VVC complexity reduction. Next, we propose a multi-stage exit CNN (MSE-CNN) model with an early-exit mechanism to determine the CU partition, in accord with the flexible QTMT structure at multiple stages. Then, we design an adaptive loss function for training the MSE-CNN model, synthesizing both the uncertain number of split modes and the target on minimized RD cost. Finally, a multi-threshold decision scheme is developed, achieving desirable trade-off between complexity and RD performance. Experimental results demonstrate that our approach can reduce the encoding time of VVC by 44.65%-66.88% with the negligible Bjøntegaard delta bit-rate (BD-BR) of 1.322%-3.188%, which significantly outperforms other state-of-the-art approaches.

I. INTRODUCTION

VVC improves coding efficiency over HEVC but sharply increases intra-mode encoding complexity, motivating data-driven prediction of its QTMT-based CU partition. The paper introduces a database, multi-stage early-exit CNN, adaptive loss, and multi-threshold decision scheme for this purpose.

  • Motivation: VVC’s intra-mode encoding complexity is substantially higher than HEVC’s because its flexible QTMT partitioning requires computationally demanding searches.The paper identifies direct QTMT-based CU-partition prediction as a data-driven alternative.
  • Contributions: The CPIV database contains diverse QTMT-based CU-partition data collected from 8,000 images and 204 video sequences at four QP values.The database is intended to support learning-based VVC complexity reduction.
  • Contributions: MSE-CNN predicts CU partitions across multiple stages using an early-exit mechanism that skips redundant CU predictions.The model is designed around VVC’s flexible QTMT structure.
  • Contributions: The adaptive loss combines classification for a variable number of split modes with optimization toward lower RD cost.This training objective addresses both mode uncertainty and RD performance.

II. RELATED WORKS

Prior complexity-reduction methods for video coding use heuristic features or learned models, but existing learning-based VVC methods do not directly predict the finalized QTMT partition. The paper positions MSE-CNN as a direct QTMT-partition predictor for intra-mode VVC.

  • Evolution of partition structures: Earlier HEVC methods mainly simplify quad-tree partitioning, while VVC introduces more flexible QTBT and QTMT structures with higher complexity.This structural difference motivates methods tailored to VVC’s newer partitioning rules.
  • Complexity-reduction approaches: Complexity-reduction research is commonly divided into heuristic approaches using handcrafted encoding features and data-driven approaches learning partition decisions from data.The paper notes that data-driven methods avoid relying heavily on handcrafted feature extraction.
  • Proposed direction: The proposed MSE-CNN directly predicts QTMT-based CU partitions for intra-mode VVC, addressing a target not covered by prior deep-learning approaches.The method is presented as distinct from predictors of CU depth or QTBT partitioning.
  • Data-driven approaches: Existing VVC acceleration methods predict CU-depth ranges or outdated QTBT partitions rather than the finalized QTMT partition adopted by VVC.This distinguishes the proposed target from earlier learning-based approaches.

III. CU PARTITION DATABASE

The paper establishes the structure and data foundation for learning VVC’s multi-stage QTMT CU partition. Its CPIV database records partition labels and RD costs across diverse content, resolutions, and QP settings.

  • Overview of CU Partition: VVC’s QTMT structure supports square and rectangular CU splits, producing more possible CU sizes than HEVC.Partitioning proceeds hierarchically across stages, with stage-dependent split-mode availability.
  • Database Establishment: The CPIV database uses 204 video sequences and 8,000 images with multiple resolutions and diverse content, divided into training, validation, and test sets.The data were encoded with VVC reference software under All-Intra settings.
  • Database Establishment: Each CU label identifies one of six split modes, while recorded RD costs support training toward VVC’s RD-optimization objective.The six modes are non-splitting, quad-tree, horizontal or vertical binary-tree, and horizontal or vertical ternary-tree.
  • Database Establishment: 6,699,233 CTU samples containing more than 1 billion CUs provide the training data for MSE-CNN.The number of possible modes varies with CU size, ranging from two to six.

IV. COMPLEXITY REDUCTION FOR INTRA-MODE VVC

MSE-CNN predicts VVC CU partitions stage by stage using conditional convolutions and size-specific sub-networks, while early exits skip unnecessary predictions. The architecture incorporates CU geometry and QP information to produce stage-appropriate split-mode decisions.

  • A. MSE-CNN for Learning CU Partition: MSE-CNN predicts QTMT CU partitions in a stage-wise top-down manner instead of checking all possible CUs through brute-force bottom-up RDO.Its input is the luminance channel of a 128×128 CTU, processed through up to six stage-specific decision units.
  • A. MSE-CNN for Learning CU Partition: Conditional convolution selects feature-processing depth according to CU size, allowing the backbone to adapt its structure to different partition scales.The number of residual units is determined by the minimum axis length of the current CU.
  • A. MSE-CNN for Learning CU Partition: Size-specific sub-networks use convolutional and fully connected layers to predict one-hot split-mode vectors whose lengths range from two to six.Non-overlapping convolutions align receptive fields with possible CU locations.
  • A. MSE-CNN for Learning CU Partition: QP is normalized and supplied as an external feature so the model can learn CU partitions under different quantization parameters.A half-mask operation scales selected feature maps or vectors by the normalized QP.
  • A. MSE-CNN for Learning CU Partition: The early-exit mechanism skips redundant CU predictions, reducing MSE-CNN complexity while preserving the multi-stage partitioning process.The paper states that the resulting complexity reduction is evaluated experimentally.

B. Loss Function for Training MSE-CNN

The MSE-CNN loss addresses stage-dependent split modes, class imbalance, and differing RD costs by combining weighted cross-entropy with an RD-cost term.

  • B. Loss Function for Training MSE-CNN: Weighted cross-entropy compensates for unbalanced split-mode proportions using inverse-frequency penalties controlled by α.α=0 risks predicting only frequent modes, α=1 may ignore prior distributions, and α=0.3 was selected on the validation set.
  • B. Loss Function for Training MSE-CNN: The RD-cost loss penalizes predictions more when assigned probability and normalized RD cost are both larger.The normalized cost compares each mode's RD cost with the minimum RD cost for that CU.
  • B. Loss Function for Training MSE-CNN: The overall MSE-CNN objective combines the adaptive cross-entropy and RD-cost losses, with β adjusting the relative contribution of the RD term.Minimizing the combined loss trains the model while accounting for both classification imbalance and RD optimization.

C. Multi-threshold Decision for MSE-CNN

The multi-threshold scheme selectively skips low-confidence split modes at each MSE-CNN stage, trading encoding-time reduction against RD performance.

  • C. Multi-threshold Decision for MSE-CNN: The scheme retains only split modes whose predicted probability exceeds τ_s times the highest predicted probability, skipping the rest during RDO.Thresholds begin at Stage 2 because Stage 1 is deterministic, and τ_s controls prediction confidence.
  • C. Multi-threshold Decision for MSE-CNN: τ_s=1 selects only the highest-probability mode for minimum complexity, whereas τ_s=0 preserves full RDO checking without RD degradation.Practical settings between 0 and 1 provide a complexity–RD trade-off.
  • C. Multi-threshold Decision for MSE-CNN: The decision scheme uses stage-specific threshold combinations because prediction accuracy differs across MSE-CNN stages.The thresholds are selected through strategies intended to preserve overall prediction accuracy.

V. EXPERIMENTAL RESULTS

The experiments evaluate whether the proposed approach reduces intra-mode VVC complexity while maintaining RD performance, including comparisons, overhead analysis, and ablation.

  • V. EXPERIMENTAL RESULTS: The experiments assess intra-mode VVC complexity reduction and RD performance against two state-of-the-art approaches.The evaluation includes a comparison with prior methods, complexity-overhead analysis, and an ablation study.
  • V. EXPERIMENTAL RESULTS: The study separately analyzes the approach's complexity overhead time and conducts an ablation study.These analyses complement the main complexity and RD-performance comparison.

A. Configuration and Settings

Experiments use VTM 7.0 on 800 images and 22 video sequences at four QP values, with separately trained MSE-CNN models for CU sizes and color channels.

  • Configuration of experiments: 800 test images and 22 JVET video sequences were encoded with VTM 7.0 in AI configuration at QP values 22, 27, 32, and 37.Encoding-time savings, BD-BR, and BD-PSNR were used to evaluate complexity reduction and RD performance.
  • Settings for MSE-CNN: 19 MSE-CNN models were trained separately according to CU size and color channel.Luminance and chrominance CU partitions are determined separately by default.
  • Settings for MSE-CNN: Figure 6 depicts the training sequence for the different MSE-CNN models and their trainable components.Each block represents training for one model.
  • Configuration of experiments: The experiments include the faster, fast, and medium operating modes.These modes are listed among the evaluated settings.

B. Performance Evaluation

The proposed MSE-CNN achieves stronger complexity–RD trade-offs than competing approaches on both video sequences and images, with performance adjustable through multi-threshold values.

  • Comparative performance: 59.57%–66.88% encoding-time reduction in faster mode exceeds the 55.65%–59.14% and 51.14%–56.85% reductions of the compared approaches on video sequences.In medium mode, the method has the lowest average BD-BR increase at 1.322% and BD-PSNR degradation at 0.055 dB.
  • Comparative performance: The faster and fast modes outperform the compared approaches across encoding-time reduction, BD-BR, and BD-PSNR on video sequences.The faster mode exceeds, while the fast mode exceeds on all three metrics.
  • Complexity–RD trade-off: The proposed approach's complexity–RD curve lies below and to the left of all other approaches for both video sequences and images.The curve is produced by varying the multi-threshold values, indicating more saved encoding time at the same BD-BR or better RD performance at the same encoding time.

C. Complexity Overhead Analysis

MSE-CNN adds little computational and storage overhead while substantially reducing encoding time, with memory usage remaining more stable than the VTM anchor across resolutions.

  • Running-time overhead: 64.53% and 45.96% average encoding-time reductions are achieved in faster and medium modes, while MSE-CNN time overhead averages 3.67% for videos and 3.02% for images.The model overhead is below 5% for most resolutions.
  • Space consumption: 2.9 MB of model files is smaller than the VTM anchor's 4.2 MB encoder size.The paper notes that this is negligible relative to typical input video storage.
  • Memory usage: MSE-CNN uses 157.3–180.5 MB of memory, whereas the VTM anchor ranges from 281.9 MB to 2148.4 MB across resolutions.The reported memory-overhead ratio remains within 55.8% for all resolutions.

D. Ablation Study

Ablation results attribute the method's complexity–RD performance to its multi-stage design, RD-aware loss, and adaptive thresholds, while QP normalization improves convergence speed.

  • Single-/multi-stage CNN structure: 2.968% BD-BR saving and 0.150 dB BD-PSNR increase favor the multi-stage design over the single-stage alternative.The comparison uses SSE-CNN and MSE-CNN with the same number of trainable parameters.
  • Loss and threshold design: Adding RD cost reduces BD-BR by 0.243%, improves BD-PSNR by 0.008 dB, and saves 0.85%–2.82% encoding time across four QP values.Adaptive stage-specific thresholds further save 0.140% BD-BR and increase BD-PSNR by 0.007 dB with similar encoding time.
  • Ablation progression: Performance and complexity reduction improve stepwise from Ablation 1 through Ablation 4 as multi-stage design, RD cost, and adaptive thresholds are added.Ablation 4 corresponds to the faster mode.
  • Individual split-mode prediction: The complete MSE-CNN outperforms every individual-split-mode ablation in complexity–RD performance.The best single-mode setting cannot reach −55% time reduction and reaches up to 6% BD-BR because multi-threshold decisions are disabled.
  • QP normalization: Original QP normalization reaches similar final loss to the unnormalized setting but begins with a smaller loss and converges faster.The comparison uses loss curves for 64×64, 32×32, 16×16, and 8×8 CUs.

VI. CONCLUSION

The paper predicts intra-mode VVC QTMT CU partitions with a deep MSE-CNN and early exits, supported by a large-scale database and multi-threshold decisions. This approach accelerates encoding while trading off complexity and RD performance, and leaves inter-mode acceleration for future work.

  • The deep MSE-CNN predicts QTMT CU partitions using a large-scale database, multi-stage processing, and early exits that skip redundant checks on unused CUs.The approach combines conditional convolution, sufficiently capacitated sub-networks, and an early-exit mechanism.
  • 44.65%–66.88% lower encoding time is achieved with a 1.322%–3.188% BD-BR increase, outperforming other state-of-the-art approaches.The multi-threshold decision scheme targets a trade-off between encoding complexity and RD performance.
  • Extending deep-learning acceleration to inter-mode VVC and other encoding components, or using network acceleration and FPGA implementations, remains future work.The stated future directions include intra-angular selection, motion-vector estimation, and faster implementations.
Loading 2006.13125v3…