Source-linked AI summary
HINet: Half Instance Normalization Network for Image Restoration
Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, Chengpeng Chen
TL;DR
Low-level image restoration has not fully benefited from normalization, motivating a design that integrates Instance Normalization without discarding content information. The paper introduces HIN Blocks and the two-stage HINet, which achieve strong results across denoising, deblurring, and deraining with favorable efficiency, including first place in an NTIRE challenge.
Problem
Normalization is rarely used in low-level vision because restoration commonly uses small patches and mini-batches, making Batch Normalization statistics unstable for scale-sensitive pixel prediction.
Method
The paper applies Instance Normalization to half of a residual block’s intermediate features, preserving content through the remaining channels, and builds a two-subnetwork multi-stage HINet with cross-stage fusion and attention.
Results
HINet surpasses the state of the art on several restoration tasks, matches MPRNet on deblurring with 22.5% of its MACs and 3.3× speedup, and wins an NTIRE track with 29.70 PSNR.
Takeaways & Limitations
HIN Blocks provide a direct Instance Normalization-based design for image restoration, with reported effectiveness across models, datasets, and restoration tasks.
Abstract
from arXiv · showhide
In this paper, we explore the role of Instance Normalization in low-level vision tasks. Specifically, we present a novel block: Half Instance Normalization Block (HIN Block), to boost the performance of image restoration networks. Based on HIN Block, we design a simple and powerful multi-stage network named HINet, which consists of two subnetworks. With the help of HIN Block, HINet surpasses the state-of-the-art (SOTA) on various image restoration tasks. For image denoising, we exceed it 0.11dB and 0.28 dB in PSNR on SIDD dataset, with only 7.5% and 30% of its multiplier-accumulator operations (MACs), 6.8 times and 2.9 times speedup respectively. For image deblurring, we get comparable performance with 22.5% of its MACs and 3.3 times speedup on REDS and GoPro datasets. For image deraining, we exceed it by 0.3 dB in PSNR on the average result of multiple datasets with 1.4 times speedup. With HINet, we won 1st place on the NTIRE 2021 Image Deblurring Challenge - Track2. JPEG Artifacts, with a PSNR of 29.70. The code is available at https://github.com/megvii-model/HINet.
1 MEGVII Technology 2 Fudan University 3 Peking University
The passage lists contact information associated with the authors’ institutions.
- The listed contact addresses use the megvii.com domain.
- The addresses identify authors by the names chenliangyu, luxin, and chenchengpeng.
- The passage provides email contacts rather than describing institutional roles or affiliations.
1. Introduction
The paper addresses the limited suitability of conventional normalization for low-level vision by introducing HIN Blocks and the two-stage HINet architecture. HINet reports strong restoration results across denoising, deblurring, and deraining while reducing computation or inference time relative to MPRNet.
- Contributions: HINet introduces Half Instance Normalization Blocks that apply Instance Normalization within image-restoration networks.The paper presents HIN Block as its central building block for integrating normalization into low-level vision.
- Contributions: HINet is a two-subnetwork multi-stage architecture using stacked HIN Blocks, cross-stage feature fusion, and supervised attention between stages.These components are intended to expand receptive fields, improve feature robustness, enrich multi-scale features, and facilitate performance gains.
- Results: HINet surpasses MPRNet across denoising and deraining while matching its deblurring performance with substantially lower computation and faster inference.The reported comparisons span multiple image restoration tasks and datasets.
- Results: 0.11 dB and 0.28 dB PSNR gains over MPRNet on SIDD use 7.5% and 30% of its MACs, with 6.8× and 2.9× speedups, respectively.
- Results: 29.70 PSNR earned HINet 1st place in the NTIRE 2021 Image Deblurring Challenge—Track2, JPEG Artifacts.
2. Related Work
Prior work established normalization and multi-stage architectures in vision, but normalization was rarely used directly in low-level restoration because restoration training and prediction impose different constraints. Instance Normalization offered a related alternative, while this paper applies it directly as feature enhancement.
- Normalization in low-level computer vision tasks:: Normalization is common in high-level vision but comparatively rare in low-level computer vision tasks.
- Normalization in low-level computer vision tasks:: Small image patches and mini-batches make Batch Normalization statistics unstable during image-restoration training.
- Normalization in low-level computer vision tasks:: Image restoration performs dense, scale-sensitive pixel prediction, whereas Batch Normalization is usually helpful in scale-insensitive tasks.
- Instance Normalization: Instance Normalization has been used for style transfer and as a regularizer for transferring synthetic-noise knowledge to real-noise denoising.
- Instance Normalization: Unlike prior transfer-learning use, this paper applies Instance Normalization directly to image restoration as feature enhancement.
- Multi-stage methods: Multi-stage restoration methods decompose complex restoration into smaller subtasks handled by lightweight subnetworks.
3. Approach
HINet is a two-subnetwork image-restoration architecture built around Half Instance Normalization Blocks in its encoders. It combines multi-stage feature connections, supervised attention, and half-channel instance normalization while optimizing PSNR loss.
- HINet: HINet consists of two U-Net subnetworks, each using four downsampling and upsampling operations in an encoder-decoder architecture.The encoders use HIN Blocks, while the decoders use ResBlocks without batch normalization.
- HINet: Figure 2 depicts three HIN Block layers for simplicity, although each HINet subnetwork contains five layers.The figure also includes CSFF and SAM modules adopted from MPRNet.
- Inter-stage connections: CSFF and SAM connect the two subnetworks to aggregate multi-scale features and propagate useful current-stage features forward.CSFF transforms features with 3 × 3 convolutions, while SAM uses modified 3 × 3 convolutions with bias.
- Training objective: The network uses PSNR loss, with each subnetwork prediction combined with its input and compared against the ground truth.Xi denotes the subnetwork input, Ri its final prediction, and Y the ground truth.
- Half Instance Normalization Block: Each HIN Block normalizes half of its channels with Instance Normalization and preserves context through the other half.The block splits convolutional features, applies learnable affine IN to one part, concatenates both parts, and adds residual features to shortcut features.
4. Experiments
Experiments evaluate HINet across denoising, deblurring, deraining, and ablations, emphasizing restoration quality alongside computational efficiency. Results show strong performance against MPRNet and consistent gains from HIN across datasets, models, and normalization choices.
- Main Results: 0.11 dB PSNR improvement over MPRNet is achieved on SIDD using 7.5% of its MACs.At 30% of MPRNet’s MACs, HINet improves PSNR by 0.28 dB and runs 2.9 times faster.
- Main Results: Comparable deblurring performance to MPRNet is obtained with 22.5% of its MACs and a 3.3× speed advantage on REDS and GoPro.The REDS evaluation uses REDS-val-300, while GoPro contains 2,103 training pairs and 1,111 evaluation pairs.
- Ablation: 0.12 dB PSNR is gained by adding HIN to HINet Simple on REDS, while DMPHN gains 0.42 dB on GoPro.These experiments test HIN across different models and datasets rather than targeting state-of-the-art performance.
- Ablation: HIN improves PSNR by 0.09 dB for PRMID and 0.06 dB for CycleISP on SIDD, indicating gains across model sizes and restoration settings.The ablation uses the first-stage UNet of HINet 0.5× without encoder–decoder skip connections for faster feedback.
- Ablation: HINet Simple with IN improves PSNR by 0.03 dB over the no-normalization baseline, while HIN achieves a 0.12 dB gain.BN decreases PSNR by 0.12 dB versus baseline, and SyncBN remains slightly below baseline at 28.09 dB versus 28.11 dB.
5. Conclusion
HINet combines the Half Instance Normalization Block with a multi-stage architecture for image restoration. Figure 5 visualizes degraded inputs alongside HINet predictions across restoration tasks.
- HINet introduces the Half Instance Normalization Block and applies Instance Normalization to half of the intermediate features while preserving content information.
- The multi-stage HINet uses feature fusion and attention-guided maps between stages to improve information flow and multi-scale feature expression.
- Figure 5 pairs degraded images with HINet predictions for visual comparison across various image restoration tasks.