PaperlayerView exact revision on arXiv

Source-linked AI summary

A Scalable Vector Graphics Latent Space

Leonardo Zini, Elia Frigieri, Lorenzo Baraldi

arXiv:2608.21893v1cs.CVcs.AI

TL;DR

SVGs are symbolic and editable, but deep learning lacks a continuous, compact, semantically meaningful space for vector content. SLS learns dense path-level embeddings with a Transformer autoencoder, and its embeddings generalize across diverse downstream tasks through simple vector-space operations.

  • Problem

    Deep learning lacks a continuous, compact, semantically meaningful space for vector content, despite SVGs’ symbolic, editable representation.

    Source: §1 · p. 1

  • Method

    SLS uses a Transformer-based autoencoder and data-driven BPE tokenization to learn compact latent representations of individual SVG paths from commands, coordinates, and style attributes.

    Source: §1 · p. 3

  • Results

    SLS embeddings generalize across retrieval, captioning, and embedding-space analysis through simple vector-space operations, without task-specific architectural changes.

    Source: §1 · p. 4

  • Takeaways & Limitations

    SLS offers compact, semantically structured representations of individual paths as groundwork for future vector-native research.

    Source: §5 · p. 15

    SLS models individual paths rather than cross-path interactions, and truncates paths to 2048 characters for computational efficiency.

    Source: §11 · p. 28

Abstract

from arXiv · show

Scalable Vector Graphics are a fundamental medium for resolution-independent visual content, yet the deep learning community lacks a continuous, dense, and invertible latent space for vector representations, the kind of foundational building block that Variational Autoencoders and their descendants have long provided for raster images. We introduce SLS (SVG Latent Space), a Transformer-based autoencoder that learns compact dense representations of individual SVG paths, the atomic visual elements from which any SVG image can be composed. By modeling SVG commands, coordinate data, and visual properties within a unified BPE-based token vocabulary, SLS learns fixed-size latent representations that jointly capture structure and appearance, and can be decoded back into valid, style-consistent SVG paths with high fidelity. The resulting embedding space is robust, invertible, and structured: embeddings lie on a unit hypersphere, enabling efficient similarity search, composition, and downstream conditioning through simple vector-space operations. Finally, we demonstrate that SLS generalizes across diverse tasks reducing their FLOPs by over 150 times compared to token-based approaches, and establishing a general-purpose latent foundation for vector graphics research.

1 Introduction

SLS addresses the lack of a compact, continuous latent space for SVG content by encoding individual paths as dense representations that can be decoded back into valid, style-complete paths. The paper presents this path-level space as a basis for vector-space reasoning and downstream SVG tasks.

1 Introduction

  • Existing SVG methods either rasterize content, losing geometric structure, or process long code sequences without learning compact, continuous vector representations.Raster-based approaches discard symbolic structure, while LLM-based methods face sparse, high-dimensional sequences that strain context windows and complicate downstream tasks.Source: §1 · pp. 1–2
  • A path-level latent space offers a unified representation because SVG primitives can be losslessly converted into paths, with each path represented by one dense vector.The representation is intended to preserve geometry and style, reduce sequences from thousands of tokens to a handful of embeddings, and reconstruct the original path with high fidelity.Source: §1 · pp. 2–3
  • SLS uses a Transformer encoder-decoder and a data-driven BPE vocabulary that jointly represents SVG commands, coordinates, and style attributes.Its lightweight autoregressive decoder reconstructs a valid, style-complete path from the dense embedding; path-level inputs reduce average sequence lengths by over two orders of magnitude versus whole-image LLM-based methods.Source: §1 · pp. 3–4
  • SLS embeddings lie close to a hypersphere and can be normalized for cosine-similarity search and composition into image-level representations.A single dense vector per path supports reconstruction with high geometric and stylistic fidelity, unlike raster encoders.Source: §1 · pp. 3–4
  • Captioning with SLS reduces training FLOPs by over 150×, while path retrieval scales to hundreds of thousands of paths and outperforms raster encoders and prior SVG-specific baselines.The paper also reports robustness to Gaussian noise and angular perturbations, and generalization to retrieval, captioning, and embedding-space analysis through vector-space operations without task-specific architectural changes.Source: §1 · pp. 3–4

2 Related Works · 3 Method

SLS represents SVG paths with compact, dense embeddings that capture geometry and style, addressing the scalability limits of long token sequences. Its Transformer autoencoder supports reconstruction, unit-sphere vector operations, path retrieval, and image captioning.

2.1 SVG Representation

  • Prior SVG methods trade flexibility for fixed structure: DeepSVG omits style, specialized vectorizers struggle on arbitrary content, and hybrids retain constraints or different granularities.Neural models trained for image-to-sequence or sketch tasks can also learn implicit SVG-like geometric and structural patterns.Source: §2.1 · pp. 4–5
  • SVG language models serialize graphics as long token sequences that strain context windows, while SLS instead uses compact path-level embeddings of geometry and style.Earlier sequence approaches can require hundreds to thousands of tokens, and image-to-SVG vectorization can require even longer sequences for detailed images.Source: §2.1 · pp. 4–5

2.2 Token-based Encoder-Decoder Architectures

  • Inspired by dense embedding spaces in machine translation and next-embedding prediction, SLS replaces sparse whole-image token sequences with compact path embeddings of geometry and style.This path-level representation is intended to address the context-length limits of sequential approaches.Source: §2.2 · p. 5

3.1 Preliminaries

  • SLS decomposes compound SVG objects into paths and trains a BPE tokenizer on text sequences of commands, numeric arguments, and style attributes, using preprocessing from.The subword vocabulary reduces average sequence length while preserving geometric fidelity; the architecture overview depicts latent encoding and reconstruction.Source: §3.1 · pp. 5–6, Figure 2 · p. 6

3.2 SLS Architecture

  • The encoder uses the final-layer EOS state as a dense path embedding; a causal Transformer decoder receives it as a prefix and greedily reconstructs the deterministic, syntactically strict sequence.The encoder adds sinusoidal positional encodings, and greedy decoding selects the highest-probability token at each step.Source: §3.2 · pp. 6–7
  • Gaussian noise with σ = 1.0 regularizes latents during training, while the objective combines token cross-entropy with a KL term that stabilizes predicted distributions.The decoder reconstructs the original token sequence conditioned on the perturbed latent vector.Source: §3.2 · p. 7
  • At inference, SLS normalizes embeddings to the unit hypersphere for vector operations without losing magnitude information, then rescales them by the empirical mean norm for decoding.Unit-norm embeddings support similarity search and interpolation, while the decoder receives representations in its expected magnitude range.Source: §3.2 · pp. 7–8

3.3 Downstream Applications

  • SLS retrieves paths by cosine similarity, which reduces to a dot product for normalized vectors, and captions SVG images by projecting path embeddings into a pretrained language model as visual tokens.The captioning setup conditions autoregressive text generation on path embeddings that encode geometric and stylistic properties.Source: §3.3 · p. 8

4 Experiments

Experiments evaluate SLS on path reconstruction, captioning, retrieval, latent-space robustness, and generalization, reporting gains over SVG-native baselines and large reductions in captioning cost.

4.1 Experimental setup

  • Training uses 1.5M images and 25M paths, with a 135M-parameter encoder-decoder trained from scratch for 480k steps on four A100 GPUs.Paths exceeding 1024 tokens are filtered during preprocessing.Source: §4.1 · p. 9

4.2 Reconstruction task

  • SLS significantly outperforms zero-shot and retrained DeepSVG at image-level reconstruction; ablations favor BPE tokenization, EOS pooling, and complete paths over command tokenizers and fixed blocks.The proposed end-to-end model jointly processes style attributes and command/argument tokens, capturing geometric and visual properties.Source: §4.2 · pp. 9–10, Table 1 · p. 10, Table 2 · p. 11

4.3 SVG Image Captioning

  • Against XML, SLS gains +4.2 CLIP-Score and +29.01 BLEU5; across LLMs, it averages +2.82 CLIP-Score, +19.19 BLEU5, and +24.63 METEOR over DeepSVG.CLIP is a non-invertible upper-bound reference, unlike SLS and the SVG-native baselines.Source: §4.3 · pp. 11–12, Table 3 · p. 12
  • Replacing XML with SLS embeddings reduces average captioning context from 2432 to 15.56 tokens, yielding 156× faster context processing and 167× fewer training TFLOPs.The compact path embeddings preserve caption quality, with SLS outperforming DeepSVG and XML across captioning metrics.Source: §4.3 · p. 12, Table 4 · p. 13

4.4 Path Retrieval

  • SLS is the strongest SVG-native path retriever over DeepSVG, improving visual and syntactic fidelity, while raster encoders score higher visually but are less sensitive to vector structure.The benchmark uses 2.5k query paths and 266k document paths from the validation set.Source: Table 5 · p. 13, §4.4 · pp. 13–14

4.5 Space Robustness

  • Decoded paths remain robust to rotations up to 30 degrees and Gaussian noise up to σ = 0.02 on ℓ2-normalized embeddings.The perturbations are applied to latent representations before decoding and comparison with the original paths.Source: Figure 6 · p. 14, §4.5 · p. 14

4.6 Generalization Beyond Path-Level Reconstruction

  • On unseen UniSVG, SLS degrades only moderately while retaining perceptual and structural quality; aggregated path embeddings nearly double DeepSVG across accuracy, precision, and recall.The image-level classifier uses the 20 most frequent ColorSVG classes and limits each image to eight paths.Source: §4.6 · pp. 14–15

5 Conclusion

SLS is a Transformer-based autoencoder that learns compact, invertible latent representations of individual SVG paths. Its fixed-size embeddings capture geometric and stylistic information and support downstream tasks through vector-space operations.

5 Conclusion

  • SLS unifies path commands, coordinates, and style attributes in a data-driven BPE vocabulary to produce dense, fixed-size embeddings.This reduces sequence lengths by orders of magnitude compared with full-SVG token-based approaches.Source: §5 · p. 15
  • The learned space is robust to latent perturbations and generalizes to retrieval and captioning through simple vector-space operations.Source: §5 · p. 15
  • SLS lays groundwork for vector-native modeling by bridging symbolic vector representations with modern representation learning and large-scale foundation models.Source: §5 · p. 15

1 Method details

SLS latents have nearly constant magnitudes, enabling normalization with negligible reconstruction impact; the method trains path encodings for autoregressive reconstruction and processes paths independently at inference.

1.1 Latent Space Norm Characterization

  • Latent vectors have mean ℓ2 norm ≈214.943, standard deviation ≈1.89, and coefficient of variation below 0.01 across 1024 dimensions.The near-constant magnitudes indicate that representations concentrate on an approximate hypersphere.Source: §1.1 · p. 19
  • Normalizing latents to unit norm and scaling them back to the mean norm bounds reconstruction error by magnitude variation and has negligible impact on reconstruction quality.Rendering from transformed latents causes only a minute, imperceptible change in Chamfer Distance.Source: §1.1 · p. 19
  • Latent semantics primarily follow angular relationships rather than radial distance, unlike typical autoencoder representations where magnitude often carries semantic information.The authors note that the geometric structure merits further investigation in structured vector graphics.Source: §1.1 · p. 19

1.2 Training and inference pseudocode

  • Training uses bidirectional encoding with last-valid-token pooling, injects Gaussian latent noise, and autoregressively reconstructs paths using cross-entropy plus 0.2-weighted KL-divergence loss.The objective encourages latent compression while maintaining accurate sequence synthesis.Source: §1.2 · p. 20
  • At inference, SLS encodes and autoregressively decodes each SVG path independently, then stacks decoded paths in their original input order.The inference procedure normalizes each latent and scales it to the learned mean norm before decoding.Source: §1.2 · pp. 20–21

2 Training Details

Section 2 specifies the Transformer autoencoder’s architecture and training procedure, then describes the vision-language captioner trained with pretrained SLS representations.

2.1 SLS Training Details

  • SLS uses a 135M-parameter Transformer autoencoder with 8 encoder and decoder layers, 8 attention heads, hidden dimension 1024, and a 448-token vocabulary.The model uses a feedforward dimension of 2048 and a maximum sequence length of 1024 tokens.Source: §2.1 · p. 21
  • Training uses AdamW with a constant 1 × 10−4 learning rate after 6,000 warmup steps, while attribute-order augmentation swaps style and d attributes with probability 0.5 per path.The training configuration also uses weight decay 0.01, batch size 192, bfloat16 mixed precision, gradient clipping at norm 1.0, and early stopping with patience 5 based on validation loss; augmentation preserves rendered geometry.Source: §2.1 · p. 22

2.2 Captioner training details

  • For captioning, pretrained SLS 1024-dimensional representations are projected into Llama-3.2-1B, Gemma-2-2B, or Qwen3-0.6B, with SVGs capped at 64 paths and text at 512 tokens.The backbones share identical training configurations; optimization uses AdamW with a constant 1 × 10−4 learning rate after 2,000 warmup steps, weight decay 0.1, batch size 384, bfloat16, and early stopping patience 3 on validation loss.Source: §2.2 · p. 22

3 Tokenizer Vocabulary Size Selection

The tokenizer ablation selects a 448-token vocabulary as a balance between compression efficiency and vocabulary utilization.

3 Tokenizer Vocabulary Size Selection

  • The ablation used tokenizers trained on 4M path samples and evaluated vocabulary sizes from 128 to 1024 tokens.After selecting the best configuration, the tokenizer was trained on 35M paths.Source: §3 · p. 22
  • A 448-token vocabulary balances compression efficiency and vocabulary utilization, achieving 110 average tokens per sequence and a 2.29× compression ratio, while its top 50 tokens cover 63.6% of occurrences.Only 139 tokens (31.03% of the vocabulary) are needed to reach 80% coverage.Source: §3 · p. 22

4 Sampling techniques

Greedy decoding is best suited to SVG reconstruction, outperforming stochastic alternatives across nearly all reported metrics. The section compares greedy decoding with multinomial, Top-K, and Nucleus sampling.

Sampling Methods

  • Greedy decoding chooses the highest-probability token, while multinomial samples from the full distribution, Top-K samples from the K most probable tokens, and Nucleus sampling uses a cumulative-probability threshold p [2].The evaluation compares these three stochastic strategies with greedy decoding.Source: §4 · p. 23

Results

  • Greedy decoding achieves the highest scores on MSE-sim, DINOv2-Sim, SSIM, mIoU, BLEU5, and METEOR, and outperforms stochastic alternatives across nearly all metrics.The paper argues that deterministic token selection is essential because even a single incorrect token can cause syntactic errors and corrupt SVG output.Source: §4 · p. 23

5 Reconstruction Quality vs Path Length

SLS reconstruction quality declines as SVG path length increases, with perceptual and structural metrics degrading more than text-based similarity. More than 95% of reconstructed SVGs are syntactically valid and renderable.

5 Reconstruction Quality vs Path Length

  • The analysis groups SVGs by longest-path character length into 0–512, 512–1024, and 1024–2048 bins, and evaluates pixel- and structure-aware metrics.The dataset maximum observed path length is 2048 characters; evaluated metrics include MSE-sim, DINOv2-sim, SSIM, LPIPS, mIoU, BLEU5, and METEOR.Source: §5 · pp. 23–24
  • SSIM drops from 96.30 (0–512) to 71.72 (1024–2048), while mIoU decreases from 88.85 to 64.86 as paths grow longer.The reported declines are 24.58 and 23.99 percentage points, respectively.Source: Table 2 · p. 24, §5 · p. 24
  • For short paths, the model achieves image-based scores of MSE 97.40, SSIM 96.30, and DINO 94.37, with path-level mIoU 88.85, BLEU5 98.62, and MET 99.10.The results indicate reliable reconstruction of geometric structure and textual path representation when sequence lengths are moderate.Source: §5 · p. 24
  • LPIPS rises from 5.76 to 40.85 and DINO similarity falls from 94.37 to 57.69 across the shortest and longest path bins.By comparison, BLEU5 falls from 98.62 to 93.38 and METEOR from 99.10 to 94.78, indicating comparatively resilient textual similarity.Source: §5 · pp. 24–25

5.1 Syntactic Error Analysis

  • 15,369 of 16,159 reconstructed SVG files are valid and renderable (95.11%); 783 of 790 syntax errors stem from malformed XML.No valid files were non-visible, and no truncated sequences were detected; only 3 errors involved invalid path commands.Source: §5.1 · p. 25

6 SLS Embedding Space Analysis

SLS embeddings degrade gracefully under perturbation, remaining robust up to approximately 30° rotation and Gaussian noise with σ ≈ 0.02. Rotations affect pixel-level similarity more rapidly than structural metrics, while Gaussian noise degrades both more uniformly.

Experimental Setup

  • Rotational perturbations rotate embeddings through varying angles in the unit hypersphere’s tangent space using a random orthogonal direction, preserving embedding norm; Gaussian noise is added and renormalized onto the sphere.The tested rotations ranged from 10° to 75°; Gaussian noise used varying σ values.Source: §6 · p. 26

Evaluation Metrics

  • Decoded paths are evaluated using pixel-level MSE, SSIM, and LPIPS, plus structural IoU, BLEU5, and METEOR metrics.BLEU and METEOR compare tokenized SVG commands and style attributes, while IoU compares rasterized binary masks of rendered paths.Source: §6 · p. 26

Qualitative Observations

  • Small perturbations preserve overall structure and semantics, while larger ones alter geometry and style; above 45° rotation or σ > 0.05, deviations can produce different shapes or styles.At intermediate perturbations, geometric and stylistic changes become noticeable, but the global shape category typically remains recognizable.Source: §6 · pp. 26–27

Quantitative Trends

  • SLS remains relatively robust up to θ ≈30 and σ ≈0.02, with pixel-based metrics degrading faster than structural metrics under rotation but both degrading more uniformly under Gaussian noise.Reconstruction metrics degrade smoothly as perturbation magnitude increases; robustness at these levels supports stable representations under realistic noise.Source: §6 · p. 27

7 SLS Reconstruction Qualitatives

SLS qualitatively reconstructs diverse SVG paths with high fidelity, preserving geometry and style, while difficult paths can show localized inaccuracies without losing structural coherence or dominant shape characteristics.

7 SLS Reconstruction Qualitatives

  • Figure 1 shows SLS consistently producing high-fidelity reconstructions that preserve geometric details and stylistic attributes across diverse SVG paths.The examples span varied path complexities and structures, demonstrating robustness and generalization of the learned representations.Source: §7 · p. 27
  • On particularly intricate or irregular paths, inaccuracies are typically localized, and reconstructions remain structurally coherent while embeddings capture dominant shape characteristics.These cases indicate boundaries of the formulation when exact token-level recovery is difficult, while the representation remains stable and meaningful.Source: §7 · p. 27

8 Captioning Additional Qualitatives … 11 Limitations

Qualitative results show that SLS embeddings support SVG captioning and retrieval, while interpolation and local sampling produce smooth, plausible paths. The paper also identifies limits in path-level modeling, sequence length, and downstream objectives.

8 Captioning Additional Qualitatives

  • Across Llama 3.2 1B, Gemma 2 2B, and Qwen3 0.6B, SLS-conditioned captions align with SVG semantics, outperform DeepSVG, but trail pretrained CLIP.Captions capture object identity, structure, and stylistic cues.Source: §8 · p. 28

9 Latent Space Interpolation and Sampling

  • SLERP between path embeddings produces smooth transitions in both geometry and appearance.The paper demonstrates this with interpolation between pairs of path embeddings.Source: §9 · p. 28
  • Random samples from neighborhoods of real path embeddings decode into valid, visually plausible SVG paths.The paper presents this as evidence that the latent space is locally smooth and robust.Source: §9 · p. 28

10 Retrieval Additional Qualitatives

  • SLS yields more semantically consistent top-5 retrieval results than DINOv2, CLIP, and DeepSVG.Source: §10 · p. 28

11 Limitations

  • SLS models paths individually, not cross-path interactions, and truncates paths at 2048 characters for computational efficiency.Aggregating path embeddings into an image-level representation and handling longer or more complex sequences are future directions.Source: §11 · p. 28
  • Captioning uses a lightweight linear adapter, while reconstruction prioritizes geometric fidelity and latent compactness over semantic or contrastive objectives.More expressive cross-modal fusion and complementary semantic or contrastive objectives may improve downstream alignment.Source: §11 · p. 28

Related papers

Loading 2608.21893v1…