Boundary-Aware Segmentation Network for Mobile and Web Applications

1 INTRODUCTION

• Segmentation applications such as image editing and manipulation require accurate boundaries and fine structures, while deep models have paid less attention to spatial accuracy.The section identifies a trade-off: large-scale features carry semantic information but often come from low-resolution maps, and simple upsampling cannot recover fine structures.Source: §1 · pp. 1–2
• BASNet combines a deeply supervised encoder-decoder, a residual refinement module, and BCE, SSIM, and IoU losses to target pixel-, patch-, and map-level accuracy.The refinement module learns residuals between coarse predictions and ground truth; the hybrid loss implicitly incorporates the goal of accurate boundaries.Source: §1 · p. 2
• The authors developed AR COPY & PASTE, which transfers background-removed objects from a phone into desktop software using augmented reality, and the web-based OBJECT CUT background-removal service.OBJECT CUT accepts an image uploaded from a local machine or URL and is intended for users without image-editing experience or software.Source: §1 · p. 2
• Across six salient-object datasets, SOC, and three camouflaged-object datasets, BASNet is competitive on regional metrics and outperforms other models on boundary metrics.The evaluations compare BASNet with 25 state-of-the-art methods on the six salient-object datasets, 16 models on SOC, and 13 camouflaged-object models on three COD datasets.Source: §1 · p. 2
• This version extends the CVPR paper with theoretical explanations of the hybrid loss, broader experiments including SOC and COD, and two applications.Source: §1 · p. 2

2.1 Traditional Image Segmentation Approaches

• Watershed and graph cut, rely on regional pixel similarities and are less effective for fine structures and smooth, accurate boundaries; active contour and perceptual grouping can yield accurate boundaries but are noise-sensitive.These traditional approaches rely on handcrafted features, objective functions, and optimization algorithms.Source: §2.1 · p. 3

2.2 Patch-wise Deep Models

• Patch-wise deep models improve robustness but classify pixels independently using local features, limiting spatial accuracy and their use of larger-scale context.The section attributes this limitation to patch-wise pixel or superpixel classification.Source: §2.2 · p. 3

2.3 FCN and its Variants

• FCNs [70] and their variants improve segmentation, but direct upsampling can miss accurate structures; atrous convolutions and CRFs add memory costs or noisy boundaries.Edge-focused methods such as HED [123], RCF [29], and CASENet [130] use shallow and deep backbone features; saliency variants also explore multi-scale feature aggregation.Source: §2.3 · p. 3

2.4 Encoder-decoder Architectures

• SegNet and U-Net [98] use encoder-decoder structures with skip connections to gradually upsample features and recover details.Related variants explore altered upsampling, feature aggregation, and cascaded or nested encoder-decoder designs.Source: §2.4 · p. 3

2.5 Deep Recurrent Models

• Recurrent models use sequential steps to segment regions, aggregate context, or correct predictions, but serial connections make them relatively less time-efficient.Examples include bidirectional LSTMs in PiCANetR [68] and cascaded prediction correction in RFCN.Source: §2.5 · pp. 3–4

2.6 Deep Coarse-to-Fine Models

• Coarse-to-fine models progressively refine predictions, with examples including RefineNet, DHSNet, SRM, R3Net+ [18], and DGRL.The section reports that these models reduce the probability of overfitting and show promising accuracy improvements.Source: §2.6 · p. 4

2.7 Boundary-assisted Deep Models

• Boundary-assisted methods add boundary losses or prediction streams, whereas BASNet uses a predict-refine architecture and a hybrid loss to model dissimilarity at pixel, patch, and map levels.Examples include NLDF, C2S, BANet [104], and EGNet [140].Source: §2.7 · p. 4

3.1 Overview

• The prediction module produces coarse maps that a multi-scale residual refinement module corrects by learning residuals between the maps and ground truth.The refined output is BASNet’s final segmentation result.Source: §3.1 · p. 5, §3.3 · p. 6

3.2 Prediction Module

• The U-Net-like prediction module uses encoder-decoder features to capture global context and local detail, with decoder stages supervised against ground truth.It produces seven probability maps during training; the final, most accurate map is passed to refinement.Source: §3.2 · p. 5

3.3 Residual Refinement Module

• Coarse maps include both misaligned or blurry boundaries and uneven regional probabilities; the proposed residual encoder-decoder refines these inaccuracies.Its encoder and decoder each have four stages, with one convolutional layer per stage.Source: §3.3 · p. 5

3.4 Hybrid Loss

• The loss behaviors reflect distinct trade-offs: BCE can blur boundaries, SSIM focuses on boundaries and foreground, and IoU can miss fine structures.The fitting examples are intermediate predictions from one image-mask pair; all losses can eventually fit it perfectly through overfitting.Source: §3.4 · pp. 6–7
• The hybrid objective combines BCE, SSIM, and IoU across eight side outputs, weighting pixelwise gradients, local structure and boundaries, and foreground regions.In single-image fitting, BCE blurs boundaries, SSIM emphasizes boundaries and foreground before driving background probabilities toward zero, while IoU can omit fine structures.Source: §3.4 · pp. 6–7

• Salient object segmentation separates salient regions from their backgrounds, where targets usually have high contrast.Source: §4 · p. 7
• Camouflaged objects resemble their backgrounds, making them difficult to perceive and segment, and many have complex structures and boundaries.Source: §4 · p. 7

• Training uses PyTorch 1.4.0, an RTX Titan GPU, Adam with a 1e-4 initial learning rate, and batches of eight.Source: §4.1 · p. 7
• Images are resized to 320×320 and randomly cropped to 288×288; training continues until loss convergence, without a validation set.Source: §4.1 · p. 7
• 400k iterations take about 110 hours, while inference takes 0.015 seconds per 320×320 image, or 70 fps.Source: §4.1 · p. 7

• Weighted F-measure Fw balances precision and recall, while relaxed boundary F-measure Fb evaluates boundary quality.Source: §4.2 · p. 7
• Mean absolute error M measures average per-pixel difference between the probability map and ground truth, while mean structural measure Sα quantifies structural similarity.Source: §4.2 · p. 7
• Mean enhanced-alignment measure Eφ considers both global and local similarity.Source: §4.2 · p. 7

4.3.1 Datasets

• The salient-object evaluation trains on DUTS-TR and tests on six benchmarks; SOC adds a 600-image test set across nine attributes.Source: §4.3.1 · pp. 7–8, §4.3.5 · p. 10

4.3.2 Ablation Study

• The architecture ablation finds BASNet best among tested configurations, and the hybrid loss improves performance especially in boundary quality.Source: §4.3.2 · p. 9

4.3.3 Comparison with State-of-the-Arts

• BASNet achieves the highest relaxed boundary F-measure Fb on all six salient-object datasets and leads on nearly all metrics across five datasets.Source: §4.3.3 · pp. 9–10, Table 2 · p. 9, Table 3 · p. 9
• Qualitative comparisons cover small low-contrast targets, complex boundaries, hollow structures, and fine structures; BASNet sometimes predicts details beyond the labels.Source: Figure 7 · p. 8, §4.3.3 · p. 10

4.3.4 Failure Cases

• On SOD, BASNet can fail in very complicated scenes with no apparent salient object and when connected targets have subtle saliency differences.Source: §4.3.4 · p. 10

4.3.5 Attribute-base Analysis

• On SOC, BASNet improves boundary measure Fb by over 5% and sometimes over 10% across attributes.Source: §4.3.5 · p. 10, Table 4 · p. 11
• SOC qualitative examples span small, out-of-view, occluded, and shape-complex targets.Source: §4.3.5 · pp. 10–11, Figure 8 · p. 10

4.4 Experiments on Camouflaged Object Segmentation

• Camouflaged object segmentation is challenging because targets may have extremely low contrast, resemble their backgrounds, and have complex shapes or structures.Evaluation uses CHAMELEON, CAMO-Test and COD10K-Test; CAMO-Test is the camouflaged subset of CAMO, and COD10K includes 10,000 images across 78 categories.Source: §4.4 · p. 11, §4.4.1 · p. 11

4.4.1 Datasets

• The three test datasets cover independent-photographer examples, a camouflaged subset, and a large collection of natural-scene images; training sets match SINet for fair comparison.The datasets are CHAMELEON, CAMO-Test and COD10K-Test.Source: §4.4.1 · pp. 11–12

4.4.2 Comparison with State-of-the-Arts

• BASNet improves relax boundary F-measure F bβ over SINet by 24.0%, 8.6% and 23.5% on CHAMELEON, CAMO-Test and COD10K-Test, respectively.It also improves weighted F-measure F wβ by 12.6%, 4.0% and 12.6% on those datasets, respectively.Source: Table 5 · p. 12, §4.4.2 · p. 12
• BASNet reduces M ↓ by 50.0%, 4.0% and 25.5% on CHAMELEON, CAMO-Test and COD10K-Test, respectively, while Sα ↑ improves on two datasets but decreases by 0.3% on CAMO-Test.Sα ↑ improves by 4.5% on CHAMELEON and 3.1% on COD10K-Test; its 0.3% decrease is on CAMO-Test.Source: §4.4.2 · p. 12
• Qualitative comparisons show BASNet handling low-contrast targets, thin structures, occlusion, complicated boundaries, hollow shapes and multiple objects.The authors attribute its strong boundary scores to its ability to perceive fine structures and complicated boundaries.Source: §4.4.2 · p. 12

4.4.3 Failure Cases

• Despite outperforming other COS models and rarely producing completely incorrect results, BASNet still produces false negatives and false positives that may limit applications and degrade user experience.Figure 10 shows typical false-negative artifacts in the first row and false-positive cases in the second.Source: §4.4.3 · p. 12, Figure 10 · p. 12

5 APPLICATIONS

• BASNet underlies two real-world applications: AR COPY & PASTE and OBJECT CUT, intended as practical tools for reducing production workload.The authors cite the network’s accuracy, speed and simple architecture as reasons for developing them.Source: §5 · p. 12

5.1 Application I: AR COPY & PASTE

• AR COPY & PASTE captures a real-world item with a phone camera, removes its background, and lets users insert the target into desktop editing software by aiming the phone at the destination.The paper describes the process as an interactive transfer between physical and digital space.Source: §5.1 · p. 13

5.1.1 Workflow of AR COPY & PASTE

• The AR COPY & PASTE workflow uses BASNet to mask non-foreground pixels for a preview, then uses SIFT to locate the target position on the computer screen before importing the image.The desktop application imports the image at the computed screen coordinates.Source: §5.1.1 · pp. 13–14

5.1.3 Comparison with Other Methods

• The prototype reduces the described image-capture and masking workflow to two operations, compared with 10 or 11 for other methods.The authors say it also lets users focus on higher-level visual goals rather than pixel-level mask decisions.Source: Table 6 · p. 14, §5.1.3 · pp. 14–15

5.1.2 Implementation Details

• The prototype sends mobile photos to a remote BASNet service on GPUs; its desktop server uses screen imagery for SIFT matching and perspective transformation, then sends commands to desktop software.The implementation is divided among a Kubernetes cluster, mobile application and desktop application.Source: §5.1.2 · p. 14, Figure 14 · p. 14

5.2 Application II: OBJECT CUT

• OBJECT CUT is an online BASNet-based background-removal service exposed as an API; its pipeline validates and preprocesses requests, runs salient-object detection, postprocesses outputs, and returns a public image URL.The service is publicly available through RapidAPI and has been used by people and companies in 86 countries.Source: §5.2 · p. 15

6 CONCLUSION

• BASNet is an end-to-end boundary-aware model with a predict-refine architecture and hybrid fusion loss, designed to capture large-scale and fine structures and produce accurate boundaries.Its prediction network and refinement module form the two principal architecture components.Source: §6 · p. 15
• Experiments on six salient-object datasets, one salient-object-in-clutter dataset and three camouflaged-object datasets show competitive performance on both region-based and boundary-aware measures.Source: §6 · p. 15
• The modular network can be adapted to other tasks by replacing its prediction network or refinement module, and its two applications are presented as practical tools for reducing real-world production workload.Source: §6 · p. 15