Source-linked AI summary
DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation
Alexandre Carlier, Martin Danelljan, Alexandre Alahi, Radu Timofte
TL;DR
Vector-graphics representation learning and generation remain underexplored despite SVGs’ widespread use. DeepSVG introduces a hierarchical generative network, dataset, and manipulation framework, and reports successful reconstruction, interpolation, manipulation, and visually precise non-autoregressive font generation.
Problem
Vector-graphics representation learning and generation are comparatively underexplored despite SVGs’ widespread use and scalability.
Method
DeepSVG hierarchically separates shapes from their encoding commands, predicts shape sets non-autoregressively, and is trained with ordered or Hungarian path assignments.
Results
DeepSVG successfully reconstructs, interpolates, and manipulates complex vector graphics, while non-autoregressive settings generate visually more precise font characters.
Takeaways & Limitations
DeepSVG can serve as an animation and manipulation tool for user-drawn SVGs and as a baseline for further vector-graphics research.
Abstract
from arXiv · showhide
Scalable Vector Graphics (SVG) are ubiquitous in modern 2D interfaces due to their ability to scale to different resolutions. However, despite the success of deep learning-based models applied to rasterized images, the problem of vector graphics representation learning and generation remains largely unexplored. In this work, we propose a novel hierarchical generative network, called DeepSVG, for complex SVG icons generation and interpolation. Our architecture effectively disentangles high-level shapes from the low-level commands that encode the shape itself. The network directly predicts a set of shapes in a non-autoregressive fashion. We introduce the task of complex SVG icons generation by releasing a new large-scale dataset along with an open-source library for SVG manipulation. We demonstrate that our network learns to accurately reconstruct diverse vector graphics, and can serve as a powerful animation tool by performing interpolations and other latent space operations. Our code is available at https://github.com/alexandre01/deepsvg.
1 Introduction
DeepSVG addresses the underexplored problem of generating complex vector graphics by modeling their hierarchical structure and introducing supporting data and manipulation resources.
- 1 Introduction: Vector graphics remain comparatively underexplored for representation learning and generation despite rasterized image generation advances.SVGs are widely used because they scale from tiny icons to billboard-size graphics.
- 1 Introduction: The approach decodes draw commands in a single forward pass rather than one command at a time.This non-autoregressive design is illustrated against one-stage autoregressive architectures in Figure 2.
- 1 Introduction: DeepSVG uses a hierarchical Transformer architecture that separates high-level shapes from the low-level commands encoding them.The encoder processes shapes separately before modeling their relations, while the decoder predicts shape representations and commands.
- 1 Introduction: The paper introduces a large-scale SVG icon dataset and an open-source framework for deep-learning-based SVG manipulation.These resources are intended to facilitate further research in complex vector graphics generation.
- 1 Introduction: The authors report successful reconstruction, interpolation, and manipulation of complex vector graphics.They position the work as an early exploration of generative models for complex vector graphics.
2 Related Work
Earlier vector-graphics generation work largely focused on restricted settings, whereas DeepSVG targets complex SVG graphics using their hierarchical structure.
- 2 Related Work: Prior icon and logo generation studies mainly modeled rasterized images with generative adversarial networks.Vector-graphics work received less attention and often focused on high-level shape synthesis or sketch generation.
- 2 Related Work: SketchRNN introduced LSTM-based variational autoencoder generation for sketches, while Sketchformer used Transformers for more stable sketch interpolation.Sketchformer did not address the generation task.
- 2 Related Work: SVG-VAE generated full vector graphics with lines and Bézier curves but was limited to glyph icons without global attributes.DeepSVG instead considers the hierarchical nature of SVG images for arbitrarily complex vector graphics.
- 2 Related Work: The SVG-Icons8 dataset contains icons with similar scale and style but diverse semantic meanings, shapes, and numbers of paths.This dataset is designed to support complex vector-graphics generation beyond datasets restricted to straight lines or fonts.
3 DeepSVG
DeepSVG represents SVGs hierarchically as sets of paths composed of draw commands, then encodes and generates these structures with a two-stage Transformer-based VAE. Its training objective addresses the unordered assignment of predicted and ground-truth paths.
- 3.1 SVG Dataset and Representation: SVG-Icons8 contains 100,000 high-quality icons across 56 categories for complex vector-graphics generation and representation learning.The dataset emphasizes consistency in scale, colors, and style while retaining diverse real-world graphics.
- 3.1 SVG Dataset and Representation: An SVG is represented hierarchically as paths, with each path defined by a sequence of draw commands and fixed-length argument lists.The representation uses a selected command subset, padding paths and command sequences with invisible elements for parallel processing.
- 3.2 SVG Embedding: DeepSVG embeds command types, quantized coordinates, and command indices into a shared dE-dimensional space.Command types use a learnable matrix, coordinates are quantized to 8 bits, and index embeddings encode sequence positions.
- 3.3 Hierarchical Generative Network: The hierarchical VAE encodes each path separately, aggregates path representations into a latent vector, and decodes path attributes before independently generating draw commands.The decoder uses learned index embeddings and predicts outputs in a single forward pass rather than autoregressively.
- 3.3 Hierarchical Generative Network: DeepSVG predicts each path’s commands feed-forward, factorizing command generation and argument prediction conditioned on the latent vector.This design is contrasted with autoregressive models that predict each next command from the preceding history.
- 3.4 Training Objective: Path-level training requires assigning unordered ground-truth paths to predictions, using either lexicographic ordered assignment or Hungarian assignment.The ordered strategy sorts paths by the lexicographic order of their starting locations, while the Hungarian strategy minimizes assignment loss.
4 Experiments
Experiments evaluate DeepSVG on complex SVG icons and glyphs, showing improved reconstruction and interpolation relative to autoregressive baselines, plus latent-space manipulation and generation results.
- DeepSVG is evaluated on the SVG-Icons8 dataset, with additional glyph-generation experiments on SVG-Fonts.
- Ablation study: Feed-forward prediction substantially improves reconstruction error and interpolation quality over the autoregressive baseline.The hierarchical model with ordered assignment achieves superior human-study results and more stable, meaningful interpolations than the alternatives.
- Animation by interpolation: DeepSVG smoothly interpolates challenging SVG path translations and deformations after finetuning on user-provided keyframes, while a player-leg deformation remains a failure case.The reported animation experiments finetune on both keyframes for about 1,000 steps.
- Latent space algebra: Latent directions support global SVG manipulations, including making icons rounder or more rectangular and adding or removing paths.The directions are computed from differences between encodings of similar SVGs with differing visual semantics.
- Font generation: For SVG-Fonts, non-autoregressive models generate consistently more visually precise glyphs without selecting the best sample or applying post-processing.Because the SVG-Font dataset is simple, the hierarchical architecture provides no significant visual improvement there.
5 Conclusion
The paper demonstrates that DeepSVG can interpolate and manipulate SVG icons and positions the hierarchical network as a baseline for future vector-graphics research.
- DeepSVG successfully performs SVG icon interpolation and manipulation.The authors identify applications including vectorisation, style transfer, classification, animation, and XML generation.
Broader Impact
DeepSVG is presented as an animation and creative-support tool for user-drawn SVGs, with potential to enhance creators’ productivity.
- DeepSVG can animate user-drawn SVGs through interpolations and other latent-space operations.The paper frames this as a potential way for creators and digital artists to enhance creativity and productivity.
- The supplementary material documents preprocessing, training, architecture, filling prediction, and additional experimental results.
A SVG Representation visualization
The paper represents SVG images as padded sets of paths, each encoded by command sequences and arguments, after converting and simplifying source SVG elements into a restricted path format.
- An SVG image is represented as a set of paths, with each path encoded as a sequence of commands and arguments.Figure 10 illustrates an SVG with 2 paths and 7 commands, using tensor representations aligned with color-coded commands.
- Path and shape conversion: The representation restricts paths to six commands while preserving expressivity by converting other basic shapes and commands into supported paths.Rectangles, lines, polylines, and polygons use line commands; circles and ellipses use four elliptical-arc commands before Bézier conversion.
- Elliptical arc conversion: Elliptical arcs use radii, rotation, endpoint, and two flags to select among four possible arcs before conversion to cubic Bézier curves.The arc is parameterized by rx, ry, ϕ, fA, fS, x2, and y2; its parametric curve and derivative determine Bézier control points.
- Path simplification: Path simplification preserves sharp-angle points while simplifying sampling along smooth curves.This preprocessing is intended to simplify representation learning for inputs containing both straight lines and parametric curves.
C Additional Training details
DeepSVG’s hierarchical architecture is detailed through its encoder–decoder components and support for path-level attributes, while additional augmentation is identified as a route to greater robustness and interpolation stability.
- Data augmentation: The dataset augments each SVG with 20 random transformations, including scaling by s ∈ [0.8, 1.2] and translation components independently sampled from [−2.5, 2.5].These transformations are described as simple data augmentations.
- Data augmentation: More complex data augmentation strategies may improve robustness in shape representation learning and interpolation stability.This is presented as the authors’ belief about a possible direction for improvement.
- Architecture: The architecture comprises encoders E(1), E(2) and decoders D(2), D(1), with Transformer blocks and temporal pooling producing fixed-dimensional representations.E(1) and E(2) retrieve single dE-dimensional vectors from command and path representations, respectively.
- Path attributes: DeepSVG’s hierarchical construction supports global path-level attributes including color, dash size, stroke-width, opacity, and filling.The paper demonstrates filling as an initial step toward modeling SVG path attributes.
F Font generation
DeepSVG is evaluated for font generation using class-conditioned Transformer blocks, producing diverse samples whose glyphs remain style-consistent for a shared latent vector.
- Experimental setup: The font-generation models are trained on the SVG-Fonts dataset for 5 epochs with class-conditioned encoder and decoder Transformer blocks.A learned label embedding is added to intermediate Transformer representations.
- Results: A shared latent vector is decoded into a style-consistent set of font characters across generated glyph samples.The reported diversity includes varying boldness and degrees of italic styling.
G Random samples of icons
DeepSVG generates diverse, visually reasonable random icon samples, although generic icon generation remains more challenging than font generation and leaves substantial room for improvement.
- Results: Random latent vectors produce diverse icons that look visually reasonable.The samples are presented as evidence of generic icon generation by DeepSVG.
- Limitations: Generic icon generation is described as more challenging than font generation, with promising results but substantial scope for improvement.
H Additional results on latent space algebra
DeepSVG supports semantic manipulations in latent and path-level spaces, including translation operations, smooth shape animations, and interpolations that also expose failures on challenging icon pairs.
- Latent space algebra: Latent-space operations yield semantically meaningful SVG manipulations, and the hierarchical model extends such operations to path encodings.Path-level operations are demonstrated using encoded individual paths.
- Path-level operations: Adding or removing the difference between path encodings of translated shapes applies the same horizontal or vertical translation to arbitrary SVG paths.The difference is computed between similar shapes that differ by a horizontal or vertical translation.
- Animations: DeepSVG generates animations that handle deformation, scaling, and rotation of shapes from two user-created drawings.Three additional animations are presented.
- Interpolations: Interpolations between challenging icon pairs are often successful, but some cases produce visually less smooth results.The additional interpolation examples include both successful interpolations and failure cases.