Source-linked AI summary
Side Window Filtering
Hui Yin, Yuanhao Gong, Guoping Qiu
TL;DR
Window-centered filtering can blur edges because neighborhoods cross discontinuities. The paper proposes Side Window Filtering, which aligns a window side or corner with the processed pixel, and reports improved edge preservation and state-of-the-art performance across multiple applications, while noting higher computational cost.
Problem
Conventional center-aligned local windows can mix pixels across edges, limiting edge preservation in filtering applications.
Method
Side Window Filtering selects among multiple windows whose side or corner aligns with the target pixel, and embeds this principle in traditional linear and nonlinear filters.
Results
SWF implementations significantly improve edge preservation and provide state-of-the-art performance across image filtering applications, while colorization experiments show reduced color leakage.
Takeaways & Limitations
A simple change in window placement extends edge-preserving filtering and local-window processing to diverse computer-vision applications.
Takeaways & Limitations
SWF has the same asymptotic complexity as the original filters but higher computational cost because it evaluates multiple windows.
Abstract
from arXiv · showhide
Local windows are routinely used in computer vision and almost without exception the center of the window is aligned with the pixels being processed. We show that this conventional wisdom is not universally applicable. When a pixel is on an edge, placing the center of the window on the pixel is one of the fundamental reasons that cause many filtering algorithms to blur the edges. Based on this insight, we propose a new Side Window Filtering (SWF) technique which aligns the window's side or corner with the pixel being processed. The SWF technique is surprisingly simple yet theoretically rooted and very effective in practice. We show that many traditional linear and nonlinear filters can be easily implemented under the SWF framework. Extensive analysis and experiments show that implementing the SWF principle can significantly improve their edge preserving capabilities and achieve state of the art performances in applications such as image smoothing, denoising, enhancement, structure-preserving texture-removing, mutual-structure extraction, and HDR tone mapping. In addition to image filtering, we further show that the SWF principle can be extended to other applications involving the use of a local window. Using colorization by optimization as an example, we demonstrate that implementing the SWF principle can effectively prevent artifacts such as color leakage associated with the conventional implementation. Given the ubiquity of window based operations in computer vision, the new SWF technique is likely to benefit many more applications.
1. Introduction
Local-window filtering often centers the window on the processed pixel, but this can mix pixels across edges and blur structure. SWF instead selects side or corner-aligned windows, extending traditional filters to improve edge preservation across several applications.
- 1. Introduction: Edge-preserving filtering is needed for HDR tone mapping, detail enhancement, structure-preserving texture removal, and related image-processing applications.The paper situates SWF among global and local optimization-based edge-preserving filters.
- 1.2. Problem and Motivation: SWF restricts an edge pixel’s support region to one side of the edge, aligning a window side or corner with the pixel instead of centering it.This follows from the paper’s Taylor-expansion analysis: approximations on either side of an edge require neighbors from the corresponding side.
- 1.3. Our Contributions: Traditional box, Gaussian, median, bilateral, and guided filters can be implemented under SWF, significantly improving their edge-preserving capabilities.The framework changes the window placement while retaining the underlying filtering families.
- 1.3. Our Contributions: SWF-based traditional filters provide state-of-the-art performance for smoothing, denoising, enhancement, texture removal, mutual-structure extraction, and HDR tone mapping.The paper presents this as a result across multiple real-world applications rather than a single filtering task.
- 1.3. Our Contributions: The SWF principle also extends beyond filtering: in optimization-based colorization, it can prevent artifacts such as color leakage.The extension targets applications using local windows and linear combinations of neighboring pixels.
2. Side Window Filtering Technique
Side Window Filtering aligns a local window's side or corner with each target pixel and selects the output closest to the input intensity, improving edge preservation over centered filtering.
- Side Window Filtering Technique: SWF restricts each support region to one side of a potential edge by aligning a window side or corner with the target pixel.The method uses eight discrete windows: left, right, up, down, northwest, northeast, southwest, and southeast.
- Side Window Filtering Technique: The final SWF output is the side-window result with minimum L2 distance to the target pixel's input intensity.Each side window is filtered separately before the closest result is selected.
- Edge-preserving analysis: S-BOX better preserves vertical, horizontal, diagonal, ramp, corner, and roof edges than BOX in the reported analysis.The comparison is supported by both the tabulated outputs and the stated roof-edge inequality.
- Edge-preserving analysis: Theoretical observations show that specific side windows preserve edges according to their orientation, while seven windows outperform BOX on roof edges.Left, northwest, and southwest windows preserve ramp edges; roof edges are not completely preserved by every side window.
- Experimental comparison: On synthetic edge and corner tests, S-BOX preserves sharp transitions visually whereas BOX smooths them away.Figure 3 compares input profiles, BOX outputs, S-BOX outputs, and zoomed edge or corner regions.
- Experimental comparison: Smoothing and denoising figures compare traditional and side-window outputs using SSIM and PSNR under stated filter parameters.The supplied captions identify the layouts, parameters, and evaluation metrics but do not provide the displayed numeric values.
3. Popular Filters under the SWF Framework
The SWF framework embeds side-window selection into common linear and nonlinear filters, adapting each filter's kernel or operation to reduce edge crossing.
- Popular Filters under the SWF Framework: Side-window versions of Gaussian, median, bilateral, and guided filters are formed by replacing each filter's operation within side windows.The paper abbreviates these variants as S-GAU, S-MED, S-BIL, and S-GUI.
- Popular Filters under the SWF Framework: S-GAU uses half or quarter Gaussian kernels to reduce the edge blurring caused when GAU kernels cross potential edges.The side-window kernel is restricted relative to the conventional Gaussian kernel.
- Popular Filters under the SWF Framework: S-MED selects the side window whose median is closest to the input intensity, improving edge preservation over MED.This follows the SWF selection principle applied to the median operation.
- Popular Filters under the SWF Framework: S-BIL prevents diffusion from crossing edges by applying bilateral weighting within side windows.Its kernel retains bilateral geometric and photometric weighting.
- Popular Filters under the SWF Framework: S-GUI slides side windows along their aligned sides and averages the resulting windows without crossing the target pixel.For L, R, U, and D windows, 2r + 1 sliding windows are averaged.
4. Applications
Across image processing applications, side-window versions of established filters preserve edges and suppress artifacts more effectively than conventional implementations. The framework also supports structure extraction and colorization, where it preserves structures and avoids color leakage.
- Applications: Side-window filtering improves edge preservation across smoothing, denoising, enhancement, HDR tone mapping, structure-texture separation, and mutual-structure extraction.The paper evaluates side-window versions against traditional filters and methods across these applications.
- 4.2. Image denoising: Side-window versions remove noise while preserving edges, whereas BOX, GAU, MED, BIL, and GUI remove noise but blur edges.This result is shown for iterative filtering of a low-light image.
- 4.3. Image enhancement: Side-window implementations eliminate halo artifacts in image enhancement and HDR tone mapping that remain in the corresponding conventional filters.The enhancement comparison reports SSIM values on the images, while the HDR comparison focuses on zoomed-in patches.
- 4.5. Structure-preserving and texture-removing on natural image: Only IS-WGF preserves the sailing boat structure while removing wave textures; IWGF, RTV, and RGF fail to preserve the structure in the example.The comparison uses a natural image from the BSR Dataset.
- 4.6. Mutual-structure extraction: MS-WGF performs as well as MJF with 10 rather than 20 iterations, while producing smoother depth results and more thoroughly removing non-mutual facial structures.The comparison uses RGB and depth image pairs.
- 4.7. Colorization: Embedding the side-window principle in colorization avoids the color leakage present in the original optimization method.The implementation replaces each centered neighborhood with a suitable side window while leaving other algorithmic aspects intact.
5. Complexity Analysis
Side-window filters retain the original filters’ complexity order but require more computation because multiple windows are evaluated.
- Side-window filters have the same order of complexity as their original filters.
- SWF increases computational cost because calculations are performed over multiple windows.
- Code optimization and GPU programming can significantly improve the computational speed of side-window filters.
6. Conclusion
The conclusion presents SWF as a general strategy that aligns window sides or corners with processed pixels to improve edge preservation. It also extends the principle beyond filtering to local-window problems such as colorization.
- SWF aligns a window side or corner with the processed pixel and significantly improves the edge-preserving capabilities of traditional filters.
- SWF extends to colorization by optimization, where it can improve performance and avoid artifacts such as color leakage.
- The principle is relevant to many computer-vision and machine-learning applications because window-based operations are extensively used there.