PaperlayerView exact revision on arXiv

Source-linked AI summary

A Generalized Framework for Edge-preserving and Structure-preserving Image Smoothing

Wei Liu, Pingping Zhang, Yinjie Lei, Xiaolin Huang, Jie Yang, Ian Reid

arXiv:1907.09642v4cs.GRcs.CVeess.IV

TL;DR

Image-smoothing tasks require different, sometimes contradictory behaviors, but most operators have fixed smoothing natures. The paper proposes a flexible truncated-Huber optimization framework and an efficient numerical solution with theoretically guaranteed convergence. Experiments across applications demonstrate its effectiveness, and the authors report state-of-the-art performance in several tasks.

  • Problem

    Many smoothing operators have fixed natures and cannot meet the different or contradictory requirements of applications such as edge-preserving and structure-preserving smoothing.

    Source: Introduction · pp. 1–2

  • Method

    The paper combines a flexible truncated Huber penalty with a non-convex, non-smooth optimization framework and provides an efficient numerical solution with theoretically guaranteed convergence.

    Source: Introduction · p. 2

  • Results

    Experiments across multiple applications demonstrate effectiveness, and the authors report outperforming specially designed approaches in many tasks with state-of-the-art performance.

    Source: Introduction · p. 2, Conclusion · p. 8

  • Takeaways & Limitations

    Different parameter settings let the framework achieve varied, even contradictory, smoothing behaviors across the four application groups.

    Source: Introduction · p. 2, Conclusion · p. 8

Abstract

from arXiv · show

Image smoothing is a fundamental procedure in applications of both computer vision and graphics. The required smoothing properties can be different or even contradictive among different tasks. Nevertheless, the inherent smoothing nature of one smoothing operator is usually fixed and thus cannot meet the various requirements of different applications. In this paper, a non-convex non-smooth optimization framework is proposed to achieve diverse smoothing natures where even contradictive smoothing behaviors can be achieved. To this end, we first introduce the truncated Huber penalty function which has seldom been used in image smoothing. A robust framework is then proposed. When combined with the strong flexibility of the truncated Huber penalty function, our framework is capable of a range of applications and can outperform the state-of-the-art approaches in several tasks. In addition, an efficient numerical solution is provided and its convergence is theoretically guaranteed even the optimization framework is non-convex and non-smooth. The effectiveness and superior performance of our approach are validated through comprehensive experimental results in a range of applications.

Introduction

Image-smoothing tasks demand different, sometimes contradictory behaviors, while most operators have a fixed smoothing nature. The paper introduces a flexible truncated-Huber optimization framework intended to handle these tasks, with a theoretically convergent numerical solution.

Introduction

  • Detail enhancement and HDR tone mapping smooth small details while preserving strong edges without blurring or sharpening them.Blurring can cause halos, while sharpening can lead to gradient reversals.Source: Introduction · p. 1
  • Clip-art artifact removal smooths small details while sharpening strong edges, whose amplitudes must not be reduced.The recovered clip-art edges may have been blurred in the compressed image.Source: Introduction · p. 1
  • Guided filtering must handle structure inconsistency between guidance and target images, which can cause blurred edges and texture-copy artifacts.Depth upsampling also needs sharpened edges because low-quality depth capture can blur them.Source: Introduction · p. 1
  • Texture removal smooths small structures even when their edges are strong, while preserving large structures even when their edges are weak.This scale-aware goal differs from the other groups’ focus on preserving strong edges.Source: Introduction · pp. 1–2
  • Most smoothing operators have fixed natures, leaving few able to meet the different or contradictory requirements of four application groups.For example, L0 smoothing sharpens edges but can cause gradient reversals in detail enhancement, while WLS performs well there but cannot sharpen edges or preserve structures.Source: Introduction · p. 2
  • The paper proposes a non-convex, non-smooth framework with a truncated Huber penalty whose parameter flexibility supports different and contradictory smoothing behaviors.The framework is presented as able to handle the four application groups.Source: Introduction · p. 2
  • The proposed framework has an efficient numerical solution with theoretically guaranteed convergence.Source: Introduction · p. 2
  • The authors report outperforming specially designed approaches in many tasks and achieving state-of-the-art performance.Source: Introduction · p. 2

Related Work

Prior smoothing methods address different application needs, but most have fixed smoothing behavior and cover only a few applications. The paper positions its flexible method as broader in application range and better-performing in several tasks.

Related Work

  • Bilateral filtering is an early edge-preserving method used for detail enhancement and HDR tone mapping, but can produce gradient reversals and halos.Its alternatives are reported to share similar problems.Source: Related Work · p. 2
  • Guided filtering avoids gradient reversals but may retain halos, while WLS handles these artifacts and L0 smoothing can remove low-amplitude structures while sharpening edges.Other guided-filtering approaches address structure inconsistency through joint filtering, scale maps, static guidance weights with Welsch penalties, or weighted analysis representations.Source: Related Work · p. 2
  • Structure-preserving approaches include rolling guidance, bilateral texture filtering, region covariances, relative total variation, convolutional networks, and TV-L1.The cited methods target texture smoothing or structures at different scales.Source: Related Work · p. 2
  • Most reviewed approaches are limited to a few applications by fixed smoothing natures, whereas the proposed method is reported to cover more applications and perform better in several tasks.Source: Related Work · p. 2

Our Approach

The approach uses a non-convex, non-smooth smoothing model with a truncated Huber penalty whose parameter settings enable different smoothing behaviors. It gives an iterative solution with theoretically guaranteed convergence and configures the model for varied smoothing tasks.

  • The truncated Huber penalty’s parameters let the model switch between edge-preserving and edge-sharpening behavior and support varied smoothing tasks.With a small a and b above the intensity range, it behaves like the Huber penalty; truncation can leave strong edges unpenalized.Source: Truncated Huber Penalty Function · p. 3, Model · p. 3
  • The model defines smoothed output u from input image f and guidance image g, with λ controlling overall smoothing strength.Its objective uses truncated Huber penalties for data and smoothness terms, while the guidance weight’s α controls sensitivity to edges in g.Source: Model · p. 3
  • The solver iteratively updates auxiliary variables and the output image, with a closed-form solution for each output update.The iteration uses the optimum conditions for the auxiliary variables before solving for the next u.Source: Numerical Solution · p. 4
  • The iterative procedure monotonically decreases the objective, and its convergence is theoretically guaranteed because the objective is bounded below.Source: Numerical Solution · p. 4
  • Specific parameter settings make the model approximate TV smoothing, yield WLS smoothing on the first iteration, or closely match TV-L1 smoothing.The WLS case uses α = 0.2 and g = f; the TV-L1-like case uses α = 0 with small a values and b values above the intensity range.Source: Property Analysis · p. 4
  • For texture removal, the model smooths small textures while producing sharper edges than TV-L1 smoothing.The structure-preserving settings use α = 0.5 and g = f; larger rd, rs, and λ can remove larger structures.Source: Property Analysis · pp. 4–5
  • For image detail enhancement and HDR tone mapping, small structures lose more amplitude than large ones while edges remain neither blurred nor sharpened.The described setting uses α = 0.2, g = f, and one iteration; the Figure 4 result uses λ = 20.Source: Property Analysis · p. 5
  • For clip-art restoration, combined edge- and structure-preserving settings can preserve both large structures with weak edges and small structures with strong edges while sharpening edges.The paper contrasts this with edge-preserving smoothing that removes the weak gray-center edge and structure-preserving smoothing that smooths the thin black circle.Source: Property Analysis · pp. 5–6, Figure 5 · p. 5

Applications and Experimental Results

Experiments across four applications report state-of-the-art performance on real ToF data and improvements in depth upsampling, texture removal, tone mapping, and clip-art restoration.

Experimental scope and runtime

  • For an 800 × 600 RGB image with N = 10, MATLAB runtimes range from 10.04 seconds at rd = rs = 1 to 96.73 seconds at rd = rs = 5.Experiments compare the method with state-of-the-art approaches in four applications; only four are shown due to space limits.Source: Applications and Experimental Results · p. 6

Guided depth map upsampling

  • On real ToF data, the method outperforms compared methods and achieves state-of-the-art performance; on simulated ToF data, it ranks best or second best by MAE.The real dataset uses an approximately 6.25× upsampling factor.Source: Figure 10 · p. 7, Figure 9 · p. 7, Table 2 · p. 8

HDR tone mapping

  • The tone-mapping results eliminate the halos and gradient reversals seen with competing smoothing approaches.BF and GF produce halos, L0 smoothing produces gradient reversals, and WLS and SG-WLS retain slight halos in some results.Source: Applications and Experimental Results · p. 7

Clip-art compression artifacts removal

  • For clip-art compression artifact removal, the method removes artifacts while sharpening edges.Compared methods leave compression artifacts, blur edges, or fail to preserve weak edges; region fusion also enhances blocky artifacts along strong edges.Source: Applications and Experimental Results · pp. 7–8

Image texture removal

Conclusion

The paper presents a flexible non-convex, non-smooth framework for edge- and structure-preserving image smoothing. Its truncated Huber penalty supports varied smoothing behaviors, and the proposed numerical solution is efficient with theoretically proven convergence.

Conclusion

  • A non-convex, non-smooth optimization framework combines a flexible truncated Huber penalty with a robust model for edge- and structure-preserving smoothing.Different parameter settings enable different, even contradictive, smoothing behaviors; the paper also proposes an efficient numerical solution and proves its convergence theoretically.Source: Conclusion · p. 8
Loading 1907.09642v4…