Source-linked AI summary
A Scalable Vector Graphics Latent Space
Leonardo Zini, Elia Frigieri, Lorenzo Baraldi
TL;DR
Deep learning lacks a continuous, dense, and invertible latent space tailored to symbolic SVG content. SLS introduces a Transformer autoencoder for compact path-level embeddings that unify geometry and style, can be decoded into valid SVG paths, and support downstream tasks with reduced computational cost. The authors position this representation as a foundation for vector-graphics research, while noting boundaries around cross-path modeling, sequence length, fusion, and objective design.
Problem
Deep learning lacks a continuous, compact, and invertible latent space for SVG representations that preserves their geometric and stylistic structure.
Method
SLS uses a Transformer-based autoencoder with data-driven BPE tokenization over SVG commands, coordinates, and style attributes to encode individual paths as fixed-size dense vectors.
Results
SLS learns invertible, structured embeddings that reconstruct SVG paths with high fidelity and generalize across retrieval, captioning, and analysis through simple vector-space operations.
Takeaways & Limitations
SLS provides a compact path-level representation that reduces sequence length and supports vector-native downstream applications without task-specific architectural changes.
Takeaways & Limitations
The method does not model interactions across multiple paths, truncates paths at 2048 characters, and uses lightweight fusion and reconstruction-focused objectives.
Abstract
from arXiv · showhide
Scalable Vector Graphics are a fundamental medium for resolution-independent visual content, yet the deep learning community lacks a continuous, dense, and invertible latent space for vector representations, the kind of foundational building block that Variational Autoencoders and their descendants have long provided for raster images. We introduce SLS (SVG Latent Space), a Transformer-based autoencoder that learns compact dense representations of individual SVG paths, the atomic visual elements from which any SVG image can be composed. By modeling SVG commands, coordinate data, and visual properties within a unified BPE-based token vocabulary, SLS learns fixed-size latent representations that jointly capture structure and appearance, and can be decoded back into valid, style-consistent SVG paths with high fidelity. The resulting embedding space is robust, invertible, and structured: embeddings lie on a unit hypersphere, enabling efficient similarity search, composition, and downstream conditioning through simple vector-space operations. Finally, we demonstrate that SLS generalizes across diverse tasks reducing their FLOPs by over 150 times compared to token-based approaches, and establishing a general-purpose latent foundation for vector graphics research.
1 Introduction
SLS addresses the lack of a continuous, compact, and invertible latent space for SVGs by representing individual paths as dense vectors that preserve geometry and style. Its path-level embeddings support reconstruction and downstream applications including retrieval and captioning.
- Motivation: SVG learning lacks a continuous, compact, semantically meaningful space that preserves symbolic geometry and supports scalable downstream use.Existing approaches rasterize SVGs or serialize them as long token sequences, discarding structure or creating sparse, high-dimensional representations.
- SLS approach: SLS is a Transformer-based autoencoder that maps individual SVG paths to compact latent representations and decodes them back into valid paths.A data-driven BPE tokenizer unifies commands, coordinates, and style attributes such as fill color, stroke width, and opacity.
- SLS approach: A single dense embedding captures both geometric and stylistic information while remaining invertible for high-fidelity SVG reconstruction.The learned embeddings lie on a hypersphere and provide a fixed-size representation rather than a long sequence of SVG tokens.
- Downstream applications: SLS embeddings generalize across retrieval, captioning, and embedding-space analysis through simple vector-space operations without task-specific architectural changes.The approach is evaluated on downstream tasks and is designed to support efficient similarity matching and conditioning.
- Related limitations: Prior SVG methods trade away flexibility, stylistic attributes, or scalability through rigid parameterizations, raster representations, or long markup sequences.These limitations motivate a unified path-level representation that retains symbolic structure while reducing sequence complexity.
3 Method
SLS uses a Transformer autoencoder to convert tokenized SVG paths into dense latent vectors and reconstruct valid path sequences. Unit-normalized embeddings support vector-space retrieval and conditioning of language models for SVG captioning.
- Path encoding: SLS tokenizes SVG commands, numeric arguments, and stylistic attributes into a unified path-level vocabulary.The tokenizer operates on individual SVG paths and uses BPE-derived subword units.
- Path encoding: The encoder represents each delimited path sequence with the final-layer hidden state at the EOS position.Input tokens receive learned embeddings and sinusoidal positional encodings before Transformer processing.
- Reconstruction: The decoder injects the latent vector as a prefix token and autoregressively reconstructs the original token sequence with greedy argmax decoding.The deterministic decoding strategy targets syntactically valid SVG paths.
- Training: Gaussian noise with σ = 1.0 is added to latent embeddings during training to encourage robust representations.The perturbed latent vector conditions the autoregressive decoder during reconstruction.
- Training: The training objective combines token cross-entropy with a λ-weighted KL-divergence regularizer between target and predicted token distributions.The regularizer is described as stabilizing training by encouraging distributional consistency.
- Downstream applications: Unit-normalized embeddings enable cosine-similarity retrieval and projection into language-model token space for SVG image captioning.A learned projection maps path embeddings into the language model’s input space, where they condition caption generation.
4 Experiments
Experiments evaluate SLS on reconstruction, captioning, retrieval, and robustness, showing high-quality reconstruction, stronger captioning than baselines, major computational savings, and effective retrieval.
- Reconstruction: SLS is evaluated at image and path levels using perceptual, structural, token-level, and command-coherency metrics.Image-level metrics include MSE-similarity, SSIM, LPIPS, and DINOv2-Similarity; path-level metrics include BLEU, METEOR, and mIoU.
- Reconstruction: SLS significantly outperforms DeepSVG on image-level reconstruction, including after DeepSVG retraining on the authors’ dataset.The comparison respects DeepSVG’s maximum-command limitation.
- Ablations: Data-driven BPE tokenization and EOS-based pooling provide the best reconstruction trade-off, whereas command-based tokenizers and fixed-length block decomposition perform worse.The ablations cover projection size, pooling, KL regularization, tokenization, and decomposition strategy.
- Captioning: +4.2 CLIP-Score and +29.01 BLEU5 over XML demonstrate stronger captioning from dense SLS embeddings.SLS also consistently outperforms DeepSVG across metrics and language-model backbones, with average gains of +2.82 CLIP-Score, +19.19 BLEU5, and +24.63 METEOR.
- Computational efficiency: 156× context-processing speedup and 167× lower training TFLOPs result from replacing long XML sequences with compact path embeddings.Average context length decreases from 2432 to 15.56 tokens across three language-model backbones.
- Retrieval and robustness: SLS retrieves more structurally similar paths and remains robust under Gaussian noise and rotational perturbations, with only moderate degradation.The evaluation uses cosine similarity over normalized embeddings and examines reconstruction after latent perturbations.
5 Conclusion
The conclusion presents SLS as a compact, invertible latent representation for SVG paths that supports reconstruction, retrieval, captioning, and broader vector-native modeling.
- Conclusion: SLS learns compact, invertible latent representations of individual SVG paths.The representations capture geometric and stylistic information in fixed-size vectors.
- Conclusion: The latent space supports downstream retrieval and captioning through vector-space operations and generalizes beyond individual path reconstruction.The paper also describes aggregation of path embeddings for complete SVG images.
- Conclusion: SLS is positioned as groundwork for vector-native modeling that connects symbolic vector representations with representation learning and foundation models.The conclusion frames this as a basis for future research in vector graphics generation and analysis.
Supplementary Material
The supplementary material identifies the authors and their institutional affiliation.
- Authors: Leonardo Zini, Elia Frigieri, and Lorenzo Baraldi are the listed authors.The supplementary text lists three authors.
- Affiliation: The authors are affiliated with the University of Modena and Reggio Emilia, Italy.The affiliation is stated directly in the supplementary material.
- Contact: The supplementary material provides author email addresses using the unimore.it domain.The displayed contact format is {name.surname}@unimore.it.
1 Method details
SLS encodes each tokenized SVG path into a normalized latent vector and autoregressively decodes it, with training based on reconstruction losses and image reconstruction obtained by stacking paths.
- Latent-space geometry: Latent vectors naturally concentrate near a hypersphere, with mean norm µ∥z∥≈214.943 and standard deviation σ∥z∥≈1.89.The latent vectors lie in R1024, and the coefficient of variation is below 0.01.
- Normalization protocol: Normalization divides each latent vector by its ℓ2 norm, while denormalization rescales it by the mean latent norm.The protocol is expressed as z_hat = z / ∥z∥_2 and z' = µ_∥z∥ · z_hat.
- Normalization protocol: The normalization–denormalization procedure introduces negligible reconstruction impact because reconstruction error is bounded by latent-magnitude variation.Rendering transformed latents causes only a minute change in Chamfer Distance.
- Training pipeline: During training, a bidirectional encoder pools the last valid token and an autoregressive decoder reconstructs the path token by token.The training pass includes padding masks, positional encodings, latent noise injection, and causal decoding.
- Training objective: The reconstruction objective combines cross-entropy and KL-divergence losses to compress path information while supporting accurate sequence synthesis.The loss is defined between decoder logits and the original token sequence.
- Inference: At inference, paths are encoded and decoded independently, then reconstructed paths are stacked in their original input order to form the image.Each decoded path is generated sequentially from the latent representation.
2 Training Details
SLS uses Transformer-based encoder-decoder architectures for path encoding and captioning, with separate configurations for SVG reconstruction and vision-language generation.
- Architecture: SLS uses a Transformer with 1024 hidden dimensions, 8 attention heads, 8 encoder layers, and 8 decoder layers.The model has a 2048-dimensional feedforward layer, a 448-token vocabulary, a 1024-token maximum sequence length, and approximately 135M parameters.
- Optimization: Training uses AdamW with a 1 × 10^-4 learning rate, warmup, weight decay, gradient clipping, and bfloat16 mixed precision.The setup uses batch size 192 and early stopping with patience of five validation checkpoints.
- Captioning model: For captioning, SLS produces 1024-dimensional path representations that are linearly projected into the language model dimension.The language model uses sequences up to 512 tokens and is fine-tuned with Llama-3.2-1B, Gemma-2-2B, or Qwen3-0.6B backbones.
3 Tokenizer Vocabulary Size Selection
The tokenizer vocabulary study identifies 448 tokens as a balance between compression efficiency and vocabulary utilization.
- Ablation procedure: The vocabulary ablation evaluates sizes from 128 to 1024 tokens using 4M path samples before training the final tokenizer on 35M paths.Metrics include sequence length, compression ratio, top-50 concentration, and vocabulary utilization.
- Selected vocabulary: 448 tokens provides the selected balance, yielding 110 average tokens per sequence and a 2.29× compression ratio.The top 50 tokens cover 63.6% of occurrences, while 139 tokens cover 80% of the vocabulary usage.
4 Sampling techniques
SLS reconstruction is evaluated across decoding strategies and path-length regimes, with deterministic decoding favored and longer paths showing progressively lower reconstruction quality.
- Sampling techniques: Greedy decoding achieves the highest scores on MSE-sim, DINOv2-Sim, SSIM, mIoU, BLEU5, and METEOR across nearly all metrics.The paper attributes this result to the need for precise token selection and syntactic correctness in SVG reconstruction.
- Path-length evaluation: The path-length analysis partitions SVGs by maximum path length into (0,512], (512,1024], and (1024,2048] character bins.Evaluation combines pixel-based metrics with structure-aware metrics, including MSE-sim, DINOv2-sim, SSIM, LPIPS, mIoU, BLEU5, and METEOR.
- Path-length results: Short paths achieve strong reconstruction, with MSE 97.40, SSIM 96.30, DINO 94.37, mIoU 88.85, BLEU5 98.62, and MET 99.10.These results indicate strong recovery of both geometric structure and textual path representation for moderate sequence lengths.
- Path-length results: SSIM drops from 96.30 to 71.72 and mIoU from 88.85 to 64.86 as maximum path length increases to 1024–2048 characters.LPIPS rises from 5.76 to 40.85, while DINO similarity falls from 94.37 to 57.69.
- Path-length results: Text-based metrics degrade less than image-based metrics, with BLEU5 changing from 98.62 to 93.38 and METEOR from 99.10 to 94.78.The paper reports that long sequences primarily affect perceptual and fine-grained structural metrics, while textual similarity remains comparatively resilient.
- Syntactic validity: 15,369 of 16,159 generated SVG files are valid and renderable, corresponding to a 95.11% validity rate.The 4.89% error rate is attributed mainly to low-level formatting issues, and no valid file was non-visible.
6 SLS Embedding Space Analysis
SLS analyzes its latent space through controlled rotations and Gaussian perturbations, finding smooth degradation and recognizable structure under small changes.
- Qualitative behavior: Small perturbations produce smooth deformations while preserving overall structure and semantics, indicating a locally smooth latent manifold.The analysis considers both rotational and Gaussian perturbations.
- Perturbation design: Rotational perturbations vary embedding directions within the tangent space of the unit hypersphere while preserving the embedding norm.The procedure uses a random orthogonal direction obtained by Gram-Schmidt.
- Perturbation design: Gaussian noise is added to ℓ2-normalized embeddings and the result is renormalized before decoding.Each perturbed representation is decoded to an SVG path and evaluated with pixel-level and structural metrics.
- Qualitative behavior: Perturbations above θ > 45° or σ > 0.05 often produce substantial structural deviations, including different shapes or styles.These larger changes correspond to moving farther from the original point in latent space.
- Quantitative trends: Reconstruction metrics degrade gracefully, with relatively high robustness up to θ ≈30° and σ ≈0.02.Smooth IoU and BLEU5 curves suggest the embedding space lacks sharp discontinuities or mode collapse.
7 SLS Reconstruction Qualitatives
SLS consistently reconstructs diverse SVG paths with high fidelity, preserving both geometric details and stylistic attributes.
- SLS produces high-fidelity reconstructions across diverse SVG paths.The qualitative examples compare SLS with DeepSVG across varied path complexities and structures.
- Reconstructions accurately preserve geometric details and stylistic attributes.
- The examples demonstrate robustness across varied path complexities and structures.
8 Captioning Additional Qualitatives
SLS embeddings support semantically aligned captioning, smooth path interpolation, plausible local sampling, and semantically consistent retrieval, while the method remains bounded by path-level modeling and reconstruction-oriented objectives.
- Captioning: Captions conditioned on SLS embeddings remain aligned with SVG visual semantics across three language models.They capture object identity, structure, and stylistic cues.
- Captioning: SLS captions significantly outperform captions based on DeepSVG’s encoder.
- Latent-space behavior: SLERP between path embeddings produces smooth transitions in geometry and appearance.
- Latent-space behavior: Local-neighborhood sampling yields valid and visually plausible SVG paths.These results demonstrate that the learned latent space is locally smooth and robust.
- Retrieval: SLS yields more semantically consistent top-5 retrieval results than DINOv2, CLIP, and DeepSVG.
- Limitations: SLS does not explicitly model interactions across multiple paths.The method operates at the path level, leaving image-level aggregation for future work.
- Limitations: The reconstruction-based objective emphasizes geometric fidelity and latent compactness rather than complementary semantic or contrastive objectives.The authors suggest that adding such objectives could enhance downstream alignment.