Natural Image Matting via Guided Contextual Attention

• Natural image matting separates foreground from background while estimating each pixel’s foreground opacity for editing, compositing, and film post-production.Source: Introduction · p. 1
• The matting equation is ill-defined because foreground color, background color, and opacity are all unknown at each pixel.Source: Introduction · p. 1
• Affinity-based methods borrow opacity from similar known patches, whereas sampling-based methods use foreground–background sample pairs to estimate unknown pixels.Source: Introduction · pp. 1–2
• Recent learning-based methods use network priors to predict alpha mattes, while SampleNet combines learning with foreground and background inpainting.Source: Introduction · p. 2
• The proposed method propagates opacity through a guided contextual attention module that uses low-level image features to guide high-level alpha feature transmission.Source: Introduction · p. 2
• The framework can be interpreted both as affinity-based alpha propagation with a network prior and as guided inpainting of the alpha image.Source: Introduction · p. 2

• Natural image matting methods are classified into sampling-based, propagation-based, and learning-based categories.Source: Related Work · p. 2
• Sampling-based methods solve the matting combination equation by selecting foreground and background colors for pixels in the unknown region.Source: Related Work · p. 2
• Propagation methods estimate alpha mattes by transferring alpha values from foreground and background regions to pixels in the unknown area.Source: Related Work · p. 2
• Information-flow Matting combines local and nonlocal color affinities with spatial smoothness through a color-mixture flow.Source: Related Work · p. 2
• Learning-based methods became dominant with deep convolutional networks, including fully neural, generative-adversarial, sampling, and index-based matting architectures.Source: Related Work · p. 2

Baseline architecture

• The customized U-Net baseline is an encoder-decoder with stacked residual blocks that takes a cropped image patch and one-hot trimap as a six-channel input.It predicts the corresponding alpha matte and replaces GCA blocks with identity layers while omitting the image feature block.Source: Baseline Structure · p. 3

Feature fusion

• The decoder combines encoder features before upsampling, aligns their channels with a two-layer shortcut block, and forwards lower-level features for fusion.These choices target detailed texture retention while avoiding additional convolutions on encoder features.Source: Baseline Structure · p. 3

Normalization

• Spectral normalization is added to each convolutional layer alongside batch normalization to constrain the network's Lipschitz constant and stabilize training.The framework overview identifies spectral normalization and batch normalization as architectural components.Source: Baseline Structure · p. 3, Figure 2 · p. 3

Loss function

• The network uses one alpha prediction loss: absolute predicted–ground-truth alpha differences averaged over the trimap's unknown region.The unknown region is denoted U, with predicted and ground-truth alpha values defined at each position.Source: Loss Function · p. 3

Data augmentation

• Data augmentation combines foreground objects, applies resizing and affine transformations, generates trimaps by dilation and erosion, crops 512×512 unknown-centered patches, and applies HSV jitter.The dataset contains only 431 foreground objects for training, motivating extensive augmentation.Source: Data Augmentation · p. 4

Feature Flows

• Guided contextual attention uses low-level image features to guide information flow on alpha features containing opacity information.The framework separates alpha feature flow from image feature flow, analogous to constructing an affinity graph from opacity and appearance information.Source: Low-level Image Feature · pp. 4–5

Patch Correlation

• The module extracts 3 × 3 image-feature patches, reshapes them as convolutional kernels, and measures unknown-to-image-patch correlation with normalized inner products.Each patch represents appearance information at a specific position.Source: Guided Contextual Attention · p. 5

Attention Scores

• A scaled softmax converts patch correlations into guided attention scores along the candidate-patch dimension.These scores define the affinity used for subsequent propagation.Source: Guided Contextual Attention · p. 5

Region Weighting

• Unknown regions borrow features from both known and unknown patches because trimaps may contain overwhelming unknown areas and few known pixels.The method assigns different weights to known and unknown patches to accommodate this imbalance.Source: Guided Contextual Attention · p. 5

Opacity Propagation

• The guided attention scores propagate information on alpha features through deconvolution, reconstructing alpha features in the unknown area.This implements affinity-style opacity propagation within the neural network.Source: Guided Contextual Attention · p. 5

Architecture and Cost

• The attention block has complexity O(c(hw)^2), so two blocks are appended at the 64 × 64 feature stage rather than higher-resolution stages.Higher-resolution features provide more detail but increase computational cost.Source: Network with Guided Contextual Attention · p. 5

Composition-1k Testing Dataset

• The method outperforms state-of-the-art approaches on the Composition-1k testing set, and its baseline also surpasses some leading methods.Comparisons include three deep matting methods and three conventional algorithms.Source: Composition-1k Testing Dataset · p. 6
• The approach achieves better qualitative performance on different foreground objects, especially in semitransparent regions and when the unknown region is larger.The authors attribute this performance to information flow between feature patches with similar appearance features.Source: Composition-1k Testing Dataset · p. 6
• A single Nvidia GTX 1080 with 8GB memory evaluates each Composition-1k image whole without scaling, while higher-resolution attention blocks improve detailed texture performance.The attention blocks operate on feature maps at resolutions much higher than 64×64 during training.Source: Composition-1k Testing Dataset · p. 6

Alphamatting.com Benchmark dataset

• GCA Matting ranks first under Gradient Error on the alphamatting.com benchmark and is generally one of its top-performing methods.Its results under the large and user trimaps are reported as much better than other top approaches, while its overall MSE ranking is nearly the same as SampleNet.Source: Alphamatting.com Benchmark dataset · p. 6
• On the Plastic bag example, the method correctly predicts pixels around an iron wire that most previous methods misclassify.The authors connect this result to contextual information from surrounding background patches.Source: Alphamatting.com Benchmark dataset · p. 7

Visualization of Attention Map

• The attention map shows no information flow in known regions, while unknown-region patches borrow information from visually similar patches.The visualization identifies where opacity information is propagated in the input image.Source: Visualization of Attention Map · p. 7

• The method solves image matting through opacity information propagation in an end-to-end neural network.Source: Conclusions · p. 7
• The guided contextual attention module imitates affinity-based propagation in a fully convolutional manner.Source: Conclusions · p. 7
• The module transmits opacity information between alpha features under the guidance of appearance information.Source: Conclusions · p. 7
• Evaluations on the Composition-1k testing dataset and alphamatting.com dataset show the superiority of the proposed method.Source: Conclusions · p. 7