HINet: Half Instance Normalization Network for Image Restoration

• The paper proposes the Half Instance Normalization Block to integrate Instance Normalization as a building block for image restoration.The authors describe it as the first model to adopt normalization directly while achieving state-of-the-art image-restoration performance.Source: §1 · pp. 1–2
• HINet stacks HIN Blocks in each subnetwork encoder to expand receptive fields at each scale and improve feature robustness.The two-stage architecture also uses cross-stage feature fusion and supervised attention between stages to enrich multi-scale features and facilitate performance gains.Source: §1 · p. 1
• Against MPRNet, HINet reports higher denoising and deraining PSNR and comparable deblurring performance, with fewer MACs and faster inference.On SIDD, HINet exceeds MPRNet by 0.11 dB and 0.28 dB in PSNR, using 7.5% and 30% of its MACs and achieving 6.8× and 2.9× speedup, respectively. On REDS and GoPro, deblurring is comparable with 22.5% of MPRNet's MACs and 3.3× speedup; deraining exceeds it by 0.3 dB PSNR on average across multiple datasets, with 1.4× speedup.Source: §1 · pp. 1–2
• With HIN, DMPHN gains 0.42 dB in PSNR on the GoPro dataset, demonstrating the block's generalization across models and datasets.Source: §1 · p. 2
• HINet wins 1st place in the NTIRE 2021 Image Deblurring Challenge - Track2. JPEG Artifacts, with a PSNR of 29.70.Source: §1 · p. 2

2.1. Normalization in low-level computer vision tasks:

• BN is often omitted in restoration because small patches and mini-batches destabilize its statistics, while dense pixel prediction is scale-sensitive.Prior work removed BN in deblurring [31] and super-resolution [26], reflecting the mismatch between BN and image restoration discussed in [51].Source: §2.1 · p. 2
• Unlike style-transfer normalization [16] and IN-based transfer learning for real-noise denoising [20], HINet uses IN directly for feature enhancement in restoration.Adaptive IN aligns channel-wise statistics between style and content images [16]; [20] uses it as a regularizer to transfer knowledge from synthetic-noise data to real-noise denoising.Source: §2.1 · p. 2

2.2. Architectures for Image Restoration

• Single-stage restoration methods often increase capacity with complex structures [3] [60], whereas HINet uses two simple U-Nets and SAM and CSFF modules [55].Prior multi-stage designs use Gaussian-Laplacian pyramids [11], recurrent unfolding [35], hierarchical multi-patch processing [58], or two encoder-decoder subnetworks plus an original-resolution subnetwork [55].Source: §2.2 · p. 2

3.1. HINet

• Each HINet subnetwork is a U-Net with four downsampling and upsampling stages, HIN Blocks in its encoder, and ResBlocks in its decoder.Encoder features are fused into the decoder, and a final 3 × 3 convolution produces the residual image output.Source: §3.1 · pp. 2–3, Figure 2 · p. 3
• Cross-stage feature fusion aggregates transformed features between subnetworks, while supervised attention passes useful features forward and suppresses less informative ones.The authors modify SAM to use 3 × 3 convolutions with bias.Source: §3.1 · p. 3
• HINet is optimized end-to-end using PSNR as its loss metric.The passage defines inputs, predictions, and ground truth for each subnetwork stage.Source: §3.1 · p. 3

3.2. Half Instance Normalization Block

• Instance Normalization keeps normalization consistent during training and inference, motivating its use in low-level tasks where batch normalization is less commonly used.The passage attributes this to patch variance differences across mini-batches and differences between training and testing formulations; it also says IN’s added parameters and computation can be ignored.Source: §3.2 · p. 4
• HIN Blocks apply Instance Normalization to half the channels and preserve context in the other half before residual processing and shortcut addition.IN uses learnable affine parameters; the block applies a 3 × 3 convolution, concatenates the normalized and unnormalized features, and forms the residual features with another 3 × 3 convolution and two leaky ReLU layers.Source: Figure 3 · p. 3, §3.2 · p. 4

4.1. Implementation Details

• Training uses SIDD for denoising, GoPro for deblurring, Rain13k for deraining, and REDS for deblurring with JPEG artifacts; image-restoration performance is assessed with PSNR and SSIM.The general setup reports training on 256 × 256 patches with batch size 64 for 4 × 10^5 iterations, with flip and rotation augmentation.Source: §4 · p. 4, §4.1 · p. 4

4.2. Main Results

• On GoPro deblurring, HINet improves PSNR by 0.11 dB over MPRNet; across REDS and GoPro, it is comparable to MPRNet with 22.5% of its MACs and 3.3× speed advantage.The REDS evaluation uses REDS-val-300, and the GoPro evaluation uses 1,111 evaluation pairs.Source: Table 2 · p. 4, §4.2 · p. 5, Table 4 · p. 5
• On average across Test100, Rain100H, Rain100L, Test2800, and Test1200, HINet gains 0.3 dB in PSNR over MPRNet for deraining.HINet is trained on Rain13k, comprising 13,712 clean-rain image pairs.Source: §4.1 · p. 4, Table 3 · p. 5
• With 7.5% of MPRNet’s MACs, HINet gains 0.11 dB in PSNR on SIDD; at 30% of its MACs, it gains 0.28 dB and runs 2.9 times faster.The denoising evaluation trains on 320 high-resolution SIDD images and tests on 1,280 patches.Source: Table 1 · p. 4, §4.2 · p. 5, Table 4 · p. 5

4.3. Ablation

• HIN raises PSNR by 0.12 dB on HINet Simple/REDS, 0.42 dB on DMPHN/GoPro, and 0.09 dB and 0.06 dB on PRMID and CycleISP/SIDD, respectively.These ablations test HIN across different models and restoration datasets.Source: §4.3 · pp. 5–6
• On REDS, HIN beats the no-normalization baseline by 0.12 dB in PSNR; IN gains 0.03 dB, while BN drops 0.12 dB and SyncBN remains 0.02 dB below baseline.SyncBN improves PSNR by 0.1 dB over BN; the authors suggest small-batch inaccurate batch statistics explain BN’s drop.Source: §4.3 · p. 6, Table 6 · p. 6
• The HIN advantage persists from 300k to 900k REDS training iterations, and placing HIN in all encoder blocks scores highest, while adding it to decoder blocks lowers PSNR from 28.23 to 28.21 dB.The authors interpret the persistent gap as HIN raising the model’s upper limit rather than merely accelerating convergence, and recommend one HIN layer per encoder block.Source: §4.3 · pp. 6–7, Figure 4 · p. 6, Table 7 · p. 7

4.4. Extension to HINet:

• For the NTIRE 2021 JPEG-artifact deblurring challenge, widening and deepening HINet reaches 29.05 dB on REDS-val-300; test-time augmentation and ensembling raise PSNR to 29.25 dB, and both challenge phases exceed others by 0.08 dB.Flip/rotation test-time augmentation adds 0.14 dB, random-crop augmentation adds 0.05 dB, and averaging three models adds 0.01 dB.Source: §4.4 · p. 7, Table 8 · p. 7, Table 9 · p. 7

• HINet surpasses SOTA across various image-restoration tasks and won 1st place in the NTIRE 2021 Image De-blurring Challenge - Track2. JPEG Artifacts [32].Source: §5 · p. 7
• HIN Block applies Instance Normalization to half of the intermediate features while retaining content information.Source: §5 · p. 7
• HINet uses feature fusion and attention-guided maps [55] between stages to ease information flow and enhance multi-scale feature expression.Source: §5 · p. 7