tcnerv:dual-domain temporal context modeling for implicit neural video compression

1. INTRODUCTION

• Prior codecs can incur complexity from multiple modeling subnetworks, temporal INR methods may add branches or propagate mismatched features, and hybrid INRs still code embeddings independently.The cited related work notes mismatch risks under fast motion, occlusion, or scene changes.Source: §1 · p. 1
• TCNeRV addresses two gaps: prior reconstructions cannot inform intermediate decoding features, and independently coded embeddings retain temporally predictable information before quantization.The proposed framework uses context from the previous reconstructed frame in both domains.Source: §1 · pp. 1–2
• MTCF aligns previous-reconstruction features with two decoder stages and combines them through channel-wise gated residual connections; TERC predicts the current embedding and codes only its residual.MTCF targets decoding-feature distortion, while TERC exploits embedding-domain redundancy.Source: §1 · pp. 1–2
• For PSNR-oriented compression, TCNeRV reduces BD-rate by 22.06%, 66.73%, and 29.85% relative to HM, DCVC, and HiNeRV, respectively.These reductions are reported for the UVG dataset.Source: §1 · p. 2

2.1. Method Overview

• The previous reconstruction supplies causal context to a six-stage conditional decoder, while TERC predicts the current content embedding from that history.Stages 2–6 progressively restore spatial resolution; the first frame uses zero context, and encoder and decoder share the closed-loop process.Source: §2.1 · p. 2, §2.3 · p. 3

2.2. Multi-Scale Temporal-Context Fusion (MTCF)

• MTCF aligns features from the previous reconstruction at decoder Stages 3 and 4, then uses channel-wise gates to fuse context and suppress unreliable information.Stage 3 supplies layout and motion cues, while Stage 4 enhances edges and textures.Source: §2.2 · p. 2, Figure 2 · p. 3

2.3. Temporal Embedding-Residual Coding (TERC)

• TERC predicts each embedding from historical reconstruction and codes only its prediction residual, rather than independently coding full embeddings.The first frame codes the complete embedding; subsequent frames encode only temporally unpredictable information, and stop-gradient prevents cancellation between prediction and residual branches.Source: §2.3 · p. 3, Figure 3 · p. 3

3.1. Experimental Configuration

• UVG's seven 1920 × 1080 videos (600 or 300 frames) are assessed with PSNR, MS-SSIM, and Bpp-based compression measures against HNeRV-Boost and codec baselines.The setup specifies regression and compression fine-tuning configurations, including 8-bit quantization and a target bit depth of 4 bits.Source: §3.1 · pp. 3–4

3.3. Video Compression

• BD-rate falls 22.06%, 66.73%, 24.10%, 57.26%, and 29.85% versus HM, DCVC, DCVC-HEM, HNeRV-Boost, and HiNeRV; it remains +8.31% and +40.22% versus DCVC-DC and NVRC.Table 2 reports BD-rate (%) relative to comparison methods on UVG.Source: Table 2 · p. 4, §3.3 · p. 4

3.2. Video Regression

• With approximately 3M parameters, TCNeRV reaches 36.08 dB on UVG, exceeding HNeRV-Boost by 2.20 dB and HiNeRV by 0.82 dB.Larger improvements on high-motion sequences are reported as confirming the benefit of historical context.Source: §3.2 · p. 4

3.4. Ablation Study

• MTCF raises average PSNR from 33.88 to 35.96 dB for +0.01M parameters and +0.4G MACs; TERC alone adds 0.20 dB, and the full model reaches 36.08 dB.The ablation indicates MTCF provides the primary reconstruction gain, with TERC adding a modest improvement.Source: Table 3 · p. 4, §3.4 · p. 4

• With approximately 3M parameters, TCNeRV reduces BD-rate by 57.26% relative to HNeRV-Boost; future work targets long-sequence errors, scene changes, random access, and rate-model optimization.Source: §4 · p. 4