PaperlayerView exact revision on arXiv

Source-linked AI summary

VecGlypher: Unified Vector Glyph Generation with Language Models

Xiaoke Huang, Bhavul Gauri, Kam Woh Ng, Tony Ng, Mengmeng Xu, Zhiheng Liu, Weiming Ren, Zhaochong An, Zijian Zhou, Haonan Qiu, Yuyin Zhou, Sen He, Ziheng Wang, Tao Xiang, Xiao Han

arXiv:2602.21461v1cs.CL

TL;DR

Learning-based glyph generation is limited by reliance on prepared exemplars and raster-to-vector workflows, motivating a more accessible, editable approach. VecGlypher uses one multimodal language model with typography-aware preprocessing and two-stage training to emit SVG paths directly from text or images. The paper reports substantially improved cross-family OOD performance over general-purpose LLMs and specialized baselines, with image-referenced generation reaching state-of-the-art performance.

  • Problem

    Existing glyph-generation workflows rely on carefully prepared exemplar images, while general-purpose and vector-graphics LLMs typically fail to produce typographically valid glyphs.

  • Method

    VecGlypher is a multimodal decoder trained with large-scale noisy-font continuation followed by expert-tagged post-training to generate SVG path tokens from text or image conditioning.

  • Results

    VecGlypher substantially improves recognizability, geometry, and distributional fidelity over general-purpose LLMs and specialized baselines in cross-family OOD evaluations.

  • Takeaways & Limitations

    Direct vector generation lets users design glyphs with words or images while providing a scalable foundation for multimodal design tools.

Abstract

from arXiv · show

Vector glyphs are the atomic units of digital typography, yet most learning-based pipelines still depend on carefully curated exemplar sheets and raster-to-vector postprocessing, which limits accessibility and editability. We introduce VecGlypher, a single multimodal language model that generates high-fidelity vector glyphs directly from text descriptions or image exemplars. Given a style prompt, optional reference glyph images, and a target character, VecGlypher autoregressively emits SVG path tokens, avoiding raster intermediates and producing editable, watertight outlines in one pass. A typography-aware data and training recipe makes this possible: (i) a large-scale continuation stage on 39K noisy Envato fonts to master SVG syntax and long-horizon geometry, followed by (ii) post-training on 2.5K expert-annotated Google Fonts with descriptive tags and exemplars to align language and imagery with geometry; preprocessing normalizes coordinate frames, canonicalizes paths, de-duplicates families, and quantizes coordinates for stable long-sequence decoding. On cross-family OOD evaluation, VecGlypher substantially outperforms both general-purpose LLMs and specialized vector-font baselines for text-only generation, while image-referenced generation reaches a state-of-the-art performance, with marked gains over DeepVecFont-v2 and DualVector. Ablations show that model scale and the two-stage recipe are critical and that absolute-coordinate serialization yields the best geometry. VecGlypher lowers the barrier to font creation by letting users design with words or exemplars, and provides a scalable foundation for future multimodal design tools.

1 Introduction

VecGlypher reframes glyph generation as multimodal language modeling over SVG paths, removing exemplar-sheet and raster intermediates. Its typography-aware data pipeline and staged training support direct, editable vector generation with stronger reported performance than general-purpose and specialized baselines.

  • Motivation: Image-referenced pipelines require prepared exemplar glyphs, creating a bottleneck for non-experts and rapid style ideation.Designers may prefer describing a style rather than drafting a reference sheet.
  • Unified formulation: VecGlypher uses one multimodal LLM to condition on text descriptions or glyph images and autoregressively emit SVG path tokens for a target character.The tokens detokenize into valid SVG paths, supporting directly editable vector outputs.
  • Training recipe: The two-stage recipe combines continuation on 39K noisy fonts with post-training on 2.5K expert-annotated fonts.The stages target SVG syntax and long-horizon geometry, then align textual and image concepts with glyph geometry.
  • Data engineering: Typography-aware preprocessing deduplicates fonts, filters malformed or long paths, normalizes coordinates, canonicalizes SVG paths, and quantizes coordinates.These steps lower sequence complexity and reduce decoding error propagation.
  • Practical advantages: The unified formulation requires no exemplar sheet, avoids rasterization artifacts, and supports scaling through larger backbones, more data, and improved tokenization.Text alone suffices, while exemplars remain supported when available.
  • Empirical validation: VecGlypher reports cleaner and more stylistically faithful glyphs than specialized image-referenced baselines and succeeds on text-referenced tasks where other LLMs fail.Ablations identify model scale and Stage 1 continuation as important for fidelity, consistency, and out-of-distribution gains.

2 Related Works

Prior work spans image-referenced raster and vector glyph generation, text-conditioned font synthesis, and language models for generic vector graphics. VecGlypher targets the unresolved combination of high-fidelity editable glyphs, typography-specific constraints, family consistency, and both text and image references.

  • Image-referenced glyph and font generation: Image-referenced systems commonly synthesize glyph bitmaps and transfer style from exemplar images, while vector-native methods predict editable curves directly.Raster pipelines remain decoupled from vectorization, limiting editability and resolution-independent reuse.
  • VecGlypher: VecGlypher unifies text- and image-referenced vector glyph generation in one multimodal language model that directly emits SVG paths.Its pipeline contrasts with prior image-focused encoder-decoder and raster-intermediate approaches.
  • Text-referenced glyph and font generation: Text-referenced systems condition font synthesis on style tags, textual attributes, impressions, incomplete tags, or free-form descriptions.
  • Language models for vector graphics: Vector-graphics language models generate SVG programs for icons and simple illustrations but generally use small curated corpora and do not address typography-specific constraints.

3 Method

VecGlypher generates one target glyph as a direct SVG path from text style descriptions or reference images. Its method combines typography-aware preprocessing, autoregressive next-token prediction, and two-stage supervised fine-tuning on complementary font corpora.

  • Representation: Each glyph is serialized as one SVG <path> string representing drawing commands and coordinates, with one-decimal precision.The method allows absolute and relative commands and uses no other SVG attributes.
  • Data processing: Preprocessing filters coverage, path length, duplicate renderings, and ambiguous fonts before extracting, normalizing, canonicalizing, and quantizing glyph paths.Fonts lacking alphanumerics, exceeding the 0.9 path-length quantile, or duplicating pangram renderings are removed; a multimodal sanity check removes problematic renders.
  • Evaluation split: Training excludes extremely long glyphs and testing removes outlines duplicated in training to measure cross-family generalization.
  • Conditioning and decoding: Next-token cross-entropy trains the model to produce a valid SVG path, which is detokenized and optionally rendered without raster denoisers, vector post-optimizers, or simplifiers.Conservative decoding favors syntactically valid SVG, and specialized SVG tokenization adds complexity without measurable gains.
  • Conditioning and decoding: The model conditions on style text or 1–8 same-font reference images plus a target character, then autoregressively predicts SVG path tokens.Text inputs are style-tag bags, while image inputs use centered 192×192 references with uniform padding.
  • Two-stage training: Stage 1 fine-tunes on noisy text-referenced Envato data to teach SVG syntax and long-horizon geometry, while Stage 2 uses expert-tagged Google Fonts with text and image references for instruction following.Envato supplies scale and diversity; Google Fonts aligns geometry with appearance instructions.

4 Experiments

Experiments evaluate VecGlypher on cross-family OOD glyph generation using recognition, geometry, similarity, and distributional metrics. Larger models, staged training, and absolute-coordinate serialization consistently improve results, with strong gains over general LLMs and vector-font baselines.

  • Evaluation setup: Cross-family OOD evaluation reports R-ACC, Chamfer Distance, CLIP, DINO, and FID on Google Fonts test families.R-ACC, CLIP, and DINO are higher-is-better; Chamfer Distance and FID are lower-is-better.
  • Text-referenced ablations: Scaling Google-only training from 4B to 27B raises R-ACC from 73.96/66.66 to 92.81/94.91 and lowers CD from 4.29/3.75 to 2.31/1.98.The paired values correspond to relative/absolute coordinates; FID also falls from 15.57/14.69 to 5.81/3.96.
  • Text-referenced ablations: A two-stage Envato-to-Google recipe with absolute coordinates reaches R-ACC 101.0, CD 1.67, DINO 94.34, and FID 3.47.Envato-only training gives strong recognition but weaker geometry, while mixed single-stage training remains inferior in geometry and fidelity.
  • Image-referenced ablations: Image-referenced two-stage training improves over Google-only, with mixed text-and-image Google post-training reaching R-ACC 99.12, CD 1.18, CLIP 26.07, DINO 95.82, and FID 2.32.With Google-only training, 27B also improves over 4B, including CD 1.41 versus 2.11 and FID 2.84 versus 7.94.
  • Comparisons to baselines: VecGlypher-70B outperforms general LLMs, achieving R-ACC 100.4, CD 1.68, DINO 94.22, and FID 3.34 against text prompts.Relative to Claude Sonnet 4.5, it delivers approximately 2.15× higher R-ACC, 68% lower CD, and 83% lower FID.
  • Comparisons to baselines: VecGlypher-27B surpasses DeepVecFont-v2 and DualVector on image-referenced generation, with approximately 2× higher R-ACC, 92% lower CD, and 97.8% lower FID.Qualitative comparisons report closed contours, preserved thin structures, and faithful target-content adherence on unseen font families.

5 Discussions and Conclusions

The discussion attributes glyph-generation difficulty to data and scale, while identifying closed-set Latin alphanumerics as the current scope boundary. VecGlypher’s results support a unified foundation for multimodal vector design, but broader writing systems remain unresolved.

  • Discussion: LLMs that generate SVG icons often fail to produce typographically valid glyphs because their training data lack glyph programs and typographic structural rules.The discussion specifically cites closure, winding, counters, and related constraints as missing annotations.
  • Discussion: Stable quality currently appears to require approximately 30B parameters, while improved tokenization, constrained decoding, and geometric adapters may lower that operating point.The passage presents this as current evidence and a possible future direction rather than a demonstrated result.
  • Conclusion: VecGlypher unifies text- and image-conditioned vector generation in one model and achieves improved recognizability, geometry, and distributional fidelity in cross-family OOD evaluations.The conclusion also identifies model scale, absolute-coordinate serialization, and staged supervision as important for stable SVG decoding and style transfer.

A Dataset Statistics (Cont’)

Dataset statistics show concentrated input descriptions but strongly long-tailed SVG outputs, especially in noisy Envato fonts. Filtering removes pathological sequences while retaining large, diverse training corpora and excluding Envato from testing because its tags are unreliable.

  • Token-length distributions: Input style descriptions remain relatively concentrated at roughly one to two dozen tags per font across both corpora.Output lengths are measured as token counts in serialized SVG <path> strings.
  • Token-length distributions: SVG path lengths are strongly long-tailed, especially for Envato, where some glyphs span tens of thousands of tokens.The longest sequences mostly reflect malformed outlines, duplicated contours, or decorative symbols.
  • Filtering motivation: Long SVG sequences increase sequence entropy, destabilize gradients, and exacerbate decoding error accumulation, motivating length-based pruning.The pruning strategy is described as “Length by pangram.”
  • Filtered corpus: The vast majority of glyphs fall into a moderate token-length regime, supporting stable training and evaluation after preprocessing.Filtered test-set output distributions are described as significantly better behaved than the pre-filtered distributions.
  • Filtered corpus: After filtering, the corpora retain 2,497 Google Fonts and 39,497 Envato fonts, containing 157,899 and 2,495,363 glyphs respectively.Envato is excluded from testing because its tags are generally noisy and lack meaningful visual descriptions.

B Prompt Templates and Samples

VecGlypher uses strict prompt templates to separate SVG-generation instructions, font-style descriptions, target characters, and image-reference requirements for text- and image-referenced generation.

  • Prompt structure: The prompt templates separate role instructions, font style descriptions, and target glyph content.The style field uses a bag of tags, while the target is a single character from 0–9, a–z, or A–Z.
  • Output constraints: The system prompt requires complete, self-contained SVG path elements and SVG-only output.Each path must be emitted in reading order and terminated with a newline.
  • Text-referenced generation: Text-referenced prompts specify font design requirements and the target character.The template is “Font design requirements: {{FONT STYLE}}. Text content: {{GLYPH CHARACTER}}.”
  • Image-referenced generation: Image-referenced prompts require matching the provided reference images for style and metrics while specifying the target character.The image prompt replaces free-form style tags with a faithful reference-matching instruction.
  • Prompt samples: Examples instantiate the templates with varied style tags and target characters such as b, 6, and V.The samples combine descriptors for weight, style, genre, drawing quality, and mood with individual character targets.

C Additional Metrics

The supplementary evaluation adds raster, perceptual, semantic, recognition, and outline-based metrics to characterize complementary aspects of glyph quality.

  • Metric overview: The evaluation reports Relative OCR Accuracy, Chamfer Distance, CLIP similarity, DINO similarity, and FID.These metrics are used alongside additional supplementary measures.
  • Evaluation protocol: All raster-based metrics use 192x192 grayscale renderings produced by the same rasterization pipeline as the qualitative figures.
  • Recognition metrics: R-ACC(U) case-normalizes OCR outputs so case mismatches do not count as shape-level recognition errors.The main-paper R-ACC is normalized by ground-truth OCR accuracy and can slightly exceed 100.
  • Outline metrics: Chamfer Distance compares sampled SVG-outline point clouds after normalization, with variants aligning translation alone or translation plus isotropic scale.Neither variant optimizes rotation, preserving intended italic slant.
  • Raster and perceptual metrics: L2 measures mean squared pixel error, while LPIPS compares deep VGG features for perceptual distance.L2 is dense but perception-insensitive; LPIPS is intended to correlate better with human judgment.
  • Raster and perceptual metrics: PSNR summarizes global reconstruction fidelity, whereas SSIM emphasizes local luminance, contrast, and structural agreement.Both are computed on grayscale rasterizations.

D Additional Baselines

Additional baselines show that publicly available general-purpose multimodal LLMs and OmniSVG transfer poorly from general SVG or icon generation to typography.

  • Compared models: The benchmark includes Llama 3.3 70B Instruct, Gemma 3 27B IT, Qwen 3 30B A3B Instruct, and OmniSVG.The first three are open-weight multimodal assistants; OmniSVG is trained for SVG generation and image vectorization.
  • Recognition results: R-ACC and R-ACC(U) remain close to zero for all evaluated open-weight LLMs and OmniSVG.The OCR engine rarely recognizes the intended characters.
  • Geometry and image quality: CD, CD(T), and CD(ST) are an order of magnitude higher than VecGlypher’s, while L2, LPIPS, and SSIM indicate severe distortions or failed rendering.
  • Specialization comparison: OmniSVG produces particularly poor glyph results despite its SVG specialization.Many outputs are invalid paths or generic shapes unrelated to the requested character or style.
  • Interpretation: These results support the distinction between simple SVG or icon generation and typography, which requires stricter geometric, stylistic, and topological constraints.

E Additional Qualitative Results

The supplementary material directs readers to HTML pages containing qualitative comparisons, ablations, and results for both text- and image-referenced generation.

  • Supplementary results: The HTML pages provide comprehensive supplementary results.
  • Text-referenced results: The supplementary material includes ablation studies and comparisons for text-referenced vector-glyph generation.Table 10 is labeled “Text-referenced ablations on data and model size.”
  • Image-referenced results: The supplementary material includes ablations for image-referenced vector-glyph generation.Table 11 is labeled “Image-referenced ablations on data and model size.”

L2 LPIPS PSNR SSIM

The evaluation reports text-referenced comparisons with general-purpose LLMs, image-referenced comparisons with vector-font baselines, and performance results for open-weight language models and a vector graphic language model.

  • Table 12 compares text-referenced generation against general LLMs.
  • Table 13 compares image-referenced generation against vector-font baselines.
  • Table 14 reports performance for open-weight LLMs and the vector graphic LLM.
Loading 2602.21461v1…