PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Layer-wise Image Vectorization

Xu Ma, Yuqian Zhou, Xingqian Xu, Bin Sun, Valerii Filev, Nikita Orlov, Yun Fu, Humphrey Shi

arXiv:2206.04655v1cs.CV

TL;DR

Image vectorization methods can overlook layer-wise topology, and prior approaches may require redundant shapes or have limited generalization. LIVE progressively optimizes closed Bézier paths with component-wise initialization and new losses. The authors report compact, precise SVGs that surpass prior work in simplicity and layer-wise topology.

  • Problem

    Prior vectorization methods overlook raster images’ topology, and some rely on domain-specific training or heavy segmentation preprocessing that can add redundant shapes.

    Source: §1 · p. 2

  • Method

    LIVE progressively adds and optimizes closed Bézier paths in a layer-wise framework, using component-wise initialization and UDF and Xing losses.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    Experiments report precise, compact SVGs across various domains, surpassing prior works in simplicity and layer-wise topology.

    Source: §1 · p. 2

  • Takeaways & Limitations

    LIVE’s layer-wise vectorization can support clipart creation and other applications, including interpolation.

    Source: §5 · p. 8

    Layer-wise optimization is less efficient than single-pass optimization.

    Source: §4.5 · p. 8

Abstract

from arXiv · show

Image rasterization is a mature technique in computer graphics, while image vectorization, the reverse path of rasterization, remains a major challenge. Recent advanced deep learning-based models achieve vectorization and semantic interpolation of vector graphs and demonstrate a better topology of generating new figures. However, deep models cannot be easily generalized to out-of-domain testing data. The generated SVGs also contain complex and redundant shapes that are not quite convenient for further editing. Specifically, the crucial layer-wise topology and fundamental semantics in images are still not well understood and thus not fully explored. In this work, we propose Layer-wise Image Vectorization, namely LIVE, to convert raster images to SVGs and simultaneously maintain its image topology. LIVE can generate compact SVG forms with layer-wise structures that are semantically consistent with human perspective. We progressively add new bezier paths and optimize these paths with the layer-wise framework, newly designed loss functions, and component-wise path initialization technique. Our experiments demonstrate that LIVE presents more plausible vectorized forms than prior works and can be generalized to new images. With the help of this newly learned topology, LIVE initiates human editable SVGs for both designers and other downstream applications. Codes are made available at https://github.com/Picsart-AI-Research/LIVE-Layerwise-Image-Vectorization.

1. Introduction

Prior vectorization methods often miss raster images’ layer-wise topology, limiting learning and requiring extra shape primitives. LIVE progressively vectorizes raster images into layer-wise SVGs, with experiments reporting compact, precise results that surpass prior works in simplicity and topology.

1. Introduction

  • Prior image-to-vector methods overlook raster topology, often requiring redundant primitives or domain-specific training and heavy segmentation preprocessing.Segmentation can split high-contrast texture into small regions, while font- and emoji-focused methods do not generalize broadly.Source: §1 · p. 2
  • LIVE is a model-free, label-free pipeline that progressively optimizes raster-to-SVG vectorization for layer-wise representation.It uses component-wise path initialization and UDF and Xing losses to pursue topology exploration while addressing color-mean error and self-interaction.Source: §1 · p. 2
  • LIVE’s fully differentiable rendering generates layer-wise SVGs described as largely consistent with human perception.Its vector graph is optimized hierarchically, layer by layer.Source: §1 · p. 2
  • LIVE’s initialization and losses are designed to reduce curve intersections and shape distortion in generated SVGs.The losses include Self-Crossing (Xing) loss and Unsigned Distance guided Focal (UDF) loss.Source: §1 · p. 2
  • Experiments report precise, compact SVGs across various domains, surpassing prior work in simplicity and layer-wise topology.The evaluation includes image-to-vector translation and interpolation across domains.Source: §1 · p. 2

2. Related Work

Prior work spans traditional and differentiable vectorization methods, while topology-oriented approaches have used segmentation or learned shape hierarchies. LIVE is presented as a no-presegmentation, no-deep-model alternative for exploring image topology.

2.1. Rasterization and Vectorization

  • Traditional vectorization often pre-segments images; other approaches include diffusion curves and gradient meshes.Some two-stage methods regress segmented components as polygons and bezigons.Source: §2.1 · p. 2
  • Differentiable rendering has supported direct shape optimization and deep models for vectorization or vector-graph generation.Examples include wavelet-rasterized bezigons, Reynolds-transport shape gradients, and recurrent, VAE, or transformer approaches [10].Source: §2.1 · p. 2

2.2. Image Topology

  • Topology-oriented methods have relied on segmentation accuracy or produced domain-specific or unstable shape ordering; LIVE requires neither pre-segmentation nor deep models.Photo2ClipArt and related methods use segmentation, while Im2Vec has domain-specific shape ordering.Source: §2.2 · p. 3

3. LIVE: Layer-wise Image Vectorization

LIVE builds SVGs by recursively adding closed Bézier paths and optimizing them layer-wise. Component-guided initialization and UDF/Xing losses target missing components, contour fidelity, and self-interaction.

3.1. Framework

  • LIVE recursively adds closed Bézier paths and optimizes them with differentiable rendering, using a path scheduler and UDF/Xing losses to build compact, layer-wise SVGs.Bézier paths reduce the design space while allowing shape complexity to vary through the number of segments.Source: §3.1 · p. 3

3.2. Component-wise Path Initialization

  • Initialization targets the largest connected regions with missing color, using their centers of mass to seed new paths and balancing component size with color.The method thresholds pixelwise color differences, quantizes the remaining valid differences into 200 bins, and selects the largest connected component.Source: §3.2 · pp. 3–4

3.3.1 UDF Loss for Reconstruction

  • UDF weights pixelwise color error by distance to the path contour, emphasizing nearby differences and suppressing inner or distant regions to preserve target colors.For multiple paths, the method extends the loss by averaging the per-pixel distance weights across paths.Source: §3.3.1 · p. 4, Figure 2 · p. 4

3.3.2 Xing Loss for Self-Interaction Problem

  • Xing loss discourages self-interacted cubic Bézier paths by constraining the angle between the first and last control-point connections to exceed 180°.The final objective combines UDF and Xing losses, weighted by λ.Source: §3.3.2 · pp. 4–5

3.4. Datasets

  • LIVE is model-free; Emoji (134 images) probes layer-wise structure, while 153-image Pics tests compact modeling on harder clipart, icons, fonts, and varied backgrounds.The authors also evaluate LIVE on realistic photos.Source: §3.4 · p. 5

3.5. Implementation Details

  • LIVE uses four segments per path and 500 optimization iterations; in qualitative reconstruction comparisons, it achieves perfect results with four paths, and adding more paths does not degrade performance.Source: §3.5 · p. 5, Figure 5 · p. 6

4. Experiments

LIVE reconstructs images with lower MSE than DiffVG, especially with fewer paths, and represents visual components progressively. It also supports interpolation, while the authors note efficiency and representation issues that remain open.

4.1. Vectorization Quality

  • LIVE achieves much lower MSE than DiffVG, especially with fewer paths, on the Emoji and Pics datasets.The comparison evaluates MSE across varying path counts.Source: Figure 6 · p. 6, §4.1 · p. 6

4.2. Layer-wise Representation

  • On simple emoji and clipart images, LIVE can represent individual components as clear, unbroken Bézier paths.The reported examples show each component learned as a single path.Source: §4.2 · p. 6, Figure 7 · p. 6
  • For photos and natural images, LIVE learns coarse to fine and models local details better than other methods under the same path count.The authors suggest progressive learning and initialization may explain this result; new paths focus on local regions after earlier paths capture the main context.Source: §4.2 · pp. 6–7

4.3. Interpolation

  • LIVE’s ordered path topology supports reasonable interpolation by linearly interpolating control points between semantically similar generated SVGs.The figure also shows interpolation by combining LIVE with a simple VAE.Source: Figure 10 · p. 7, §4.3 · p. 8

4.4. Ablation study

  • Circle initialization reduces artifacts relative to random initialization and encourages convex closed paths, while Xing loss mitigates self-interaction under the same optimization conditions.The corresponding figures illustrate the initialization comparison and the effect of Xing loss.Source: §4.4 · p. 8, Figure 11 · p. 8, Figure 12 · p. 8

4.5. Discussion

  • Layer-wise optimization is less efficient than single-pass optimization, and gradient colors and adaptive segment and color choices remain future work.The authors also identify combining layer-wise vectorization with deep amodal segmentation as a direction for complicated images.Source: §4.5 · p. 8

5. Conclusion

LIVE progressively vectorizes raster images with a layer-wise representation, using component-wise initialization and losses for reconstruction and self-interaction. The authors report component-level results on simple images, coarse-to-fine modeling on complex images, two evaluation datasets, and an interpolation application.

5. Conclusion

  • LIVE progressively infers raster images using component-wise path initialization and UDF and Xing losses.UDF supports vectorization, while Xing mitigates self-interaction.Source: §5 · p. 8
  • LIVE explicitly vectorizes individual components in simple emoji or clipart and explores coarse-to-fine representation in complex natural images.Source: §5 · p. 8
  • The paper presents Emoji and Pics datasets for evaluation and shows that LIVE can be integrated with other methods for applications such as interpolation.Source: §5 · p. 8

A. User Study

A user study compared LIVE with DiffVG and Neural Painting on users’ judgments of layer-wise representation. Most participants selected LIVE as having the best learning progression.

A. User Study

  • In a study of 21 randomly selected Emoji and Pics images, 20 users chose the method with the best learning progression.Source: A. User Study · p. 11
  • LIVE received an average score of 73.8%, compared with 14.3% for DiffVG and 11.9% for Neural Painting.Source: A. User Study · p. 11
  • The authors report that most users judged LIVE to have the best layer-wise representation.Source: A. User Study · p. 11

B. More Qualitative Comparisons

Additional emoji examples show LIVE disentangling visual concepts as paths are added. The examples report that adding paths does not decrease performance.

B. More Qualitative Comparisons

C. Xing loss weights

Xing loss reduces self-interaction risk, but its weight must remain small: a weight of 1.0 causes optimization failure, while 0.01 is the default.

C. Xing loss weights

D. Orderness of LIVE-generated SVGs

LIVE's progressive learning and component-wise initialization produce a deterministic order of optimized Bézier paths, supporting better SVG interpolation than DiffVG.

D. Orderness of LIVE-generated SVGs

E. More Interpolation Results

LIVE supports SVG interpolation experiments, including multi-image interpolation with a VAE; the reported four-image results are similar to those of other VAE-based vectorization methods.

E. More Interpolation Results

  • Combining a VAE model with LIVE achieves results similar to other VAE-based vectorization methods in interpolation among four randomly selected images.Source: E. More Interpolation Results · p. 11
  • Figure 15 compares linear interpolation between two SVGs generated by DiffVG with random seeds, DiffVG with fixed seeds, and LIVE.Source: Figure 15 · p. 12

F. More Vectorization Quality

LIVE is compared with DiffVG and Neural Painting under matched path or stroke counts, with stronger representation quality reported particularly at low counts.

F. More Vectorization Quality

Related papers

Loading 2206.04655v1…