Source-linked AI summary
Few-shot Font Generation with Localized Style Representations and Factorization
Song Park, Sanghyuk Chun, Junbum Cha, Bado Lee, Hyunjung Shim
TL;DR
Few-shot font generation must create a font library from very few references, but universal style representations limit the capture of local detail in complex scripts such as Chinese. LF-Font uses localized, component-wise style representations and factorization, and outperforms five state-of-the-art methods in Chinese generation with eight references.
Problem
Universal style representations limit the capture of diverse local styles in complex scripts such as Chinese, whose glyphs contain many components.
Source: §1 · pp. 1–2Method
LF-Font represents styles component-wise and factorizes localized style features into style and component factors to reconstruct unseen style-component pairs.
Source: §1 · p. 2, §3.3 · p. 3Results
Takeaways & Limitations
LF-Font produces complex glyphs that preserve local detail styles and the global structure of target characters.
Source: §4.3 · p. 7, §5 · p. 8When a one-shot reference glyph is too simple to provide solid component-wise features, generated images can have poor visual quality.
Source: §4.4 · p. 8
Abstract
from arXiv · showhide
Automatic few-shot font generation is a practical and widely studied problem because manual designs are expensive and sensitive to the expertise of designers. Existing few-shot font generation methods aim to learn to disentangle the style and content element from a few reference glyphs, and mainly focus on a universal style representation for each font style. However, such approach limits the model in representing diverse local styles, and thus makes it unsuitable to the most complicated letter system, e.g., Chinese, whose characters consist of a varying number of components (often called "radical") with a highly complex structure. In this paper, we propose a novel font generation method by learning localized styles, namely component-wise style representations, instead of universal styles. The proposed style representations enable us to synthesize complex local details in text designs. However, learning component-wise styles solely from reference glyphs is infeasible in the few-shot font generation scenario, when a target script has a large number of components, e.g., over 200 for Chinese. To reduce the number of reference glyphs, we simplify component-wise styles by a product of component factor and style factor, inspired by low-rank matrix factorization. Thanks to the combination of strong representation and a compact factorization strategy, our method shows remarkably better few-shot font generation results (with only 8 reference glyph images) than other state-of-the-arts, without utilizing strong locality supervision, e.g., location of each component, skeleton, or strokes. The source code is available at https://github.com/clovaai/lffont.
1 Introduction
Universal style representations limit the capture of local styles in complex, glyph-rich scripts. LFFont uses localized style representations and factorization to generate Chinese fonts from few references, outperforming prior methods in the reported evaluation.
1 Introduction
- Font design is labor-intensive and expertise-dependent, motivating automatic generation of complete glyph sets from a few reference images.The motivation is especially pertinent to glyph-rich scripts such as Chinese.Source: §1 · p. 1
- The generation scenario uses many paired examples for training but only a few unseen-style references at generation, without additional finetuning.The scenario is relevant when target-style glyphs are expensive to collect or finetuning resources are limited.Source: §1 · p. 1
- Universal style representations limit diverse local-style modeling, a particular concern for scripts such as Chinese, whose characters have varying numbers of complex components.Character visual quality is sensitive to local damage and distinctive component-wise styles.Source: §1 · p. 1
- DM-Font targets scripts with fixed component counts and positions, while Chinese has varying component structures and hundreds of components that few references cannot cover.The passage reports that DM-Font fails to disentangle complex Chinese glyph structures and requires every component in the reference set.Source: §1 · pp. 1–2
- LFFont combines character-wise localized style representations with component and style factors to generate vocabularies without observing every component or using strong locality supervision.The cited locality supervision examples include component locations, skeletons, and strokes.Source: §1 · p. 2
- With eight reference glyphs, LFFont outperforms five state-of-the-art methods by significant gaps across various evaluation metrics.The evaluation focuses on Chinese few-shot font generation.Source: §1 · p. 2
2 Related Works
Prior work applies image-to-image translation and style-content disentanglement to font generation, while other approaches rely on component compositionality. Existing methods nevertheless have limits in complex scripts, local-style capture, and few-shot settings.
2 Related Works
- Image-to-image translation maps source to target domains while preserving source content, and has been adopted for font generation, including paired translation from a fixed source font.The cited examples include Isola et al. (2017), Zhu et al. (2017), and Tian (2017).Source: §2 · p. 2
- Most few-shot approaches disentangle style and content, while DM-Font instead decomposes glyphs into stylized components and reassembles them using strong compositionality priors.AGIS-Net is cited as using a local texture discriminator and local texture refinement loss.Source: §2 · p. 2
- Prior few-shot font-generation methods can fail on complex glyph-rich scripts, miss diverse local styles, or lose complex content structures; this work targets these limitations.The passage associates these limitations with prior methods including [Azadi et al. 2018], [Sun et al. 2018; Zhang, Zhang, and Cai 2018; Gao et al. 2019; Liu et al. 2019; Srivatsan et al. 2019], and [Cha et al. 2020a,b].Source: §2 · p. 2
- Other Chinese font-generation methods may require many references or finetuning, or target fixed-style or unpaired settings rather than this few-shot scenario.SCFont and ChiroGAN use skeleton or stroke extraction and require many references; the passage gives 775 as an example for Jiang et al. (2019).Source: §2 · p. 2
3 Few-shot Font Generation with Localized Style Representations and Factorization
LF-Font uses localized component-wise style representations and factorization modules to generate glyphs from few references. Its pipeline combines these with source-glyph content representations.
3.1 Problem definition
- Few-shot generation targets every character in an unseen style using very few references, such as eight, with separate style and content representations.The generator uses references for style and a source glyph for character content.Source: §3.1 · p. 3
3.2 Localized style representations
- Component-wise style features capture local styles through a character’s component set, but directly learning them requires too many references for Chinese.The passage gives 229 as an example of the total number of Chinese components.Source: §3.2 · p. 3
3.3 Completing missing localized style representations by factorization modules
- Factorization decomposes component-wise style features into style and component factors, allowing missing style-component pairs to be reconstructed from partial references.The factors are combined element-wise; consistency loss trains factors for the same style or component to agree across glyphs.Source: §3.3 · pp. 3–4
3.4 Generation
- LF-Font combines reference-derived style factors with source-derived component factors, while a content encoder preserves global character structure for generation.The generator uses localized style and global content representations, without strong localization supervision of the source glyph.Source: §3.4 · p. 4
3.5 Training
- Training first uses directly extracted component-wise style features, then adds factorization modules and jointly trains the model with adversarial, reconstruction, feature-matching, and component-classification objectives.The full objective combines the stated losses with hyperparameters controlling their contributions.Source: §3.5 · p. 5
4 Experiments
The experiments evaluate LF-Font on Chinese few-shot font generation against prior methods and analyze localized style and content encoding. Results report stronger overall performance, while ablations show that localized style representation and content encoding contribute distinct benefits.
4.1 Datasets and evaluation metrics
- Evaluation uses 467 fonts for training and 15 unseen fonts; models are separately evaluated on 2,615 seen and 280 unseen characters. Glyph quality is assessed with LPIPS, style-aware and content-aware classifier accuracies, and FID.Source: §4.1 · pp. 5–6
4.3 Experimental results
- For unseen characters, LF-Font achieves a 28.7pp higher harmonic mean accuracy than FUNIT and a 3.4 lower harmonic mean FID than AGIS-Net.The evaluation uses eight reference glyphs per style and averages 50 runs with different reference characters.Source: Table 2 · p. 6, §4.3 · p. 7
4.2 Comparison methods
- The comparison covers five state-of-the-art methods, categorized by whether they explicitly model style representations or content representations.The methods are evaluated in the Chinese few-shot font-generation setting.Source: §4 · p. 5, §4.2 · p. 6
4.3 Experimental results
- Qualitatively, AGIS-Net drops local details, FUNIT can damage local structure, and DM-Font can muddle component locations or generate the wrong component.LF-Font is described as preserving both component-level detail and the target character's global structure.Source: §4.3 · p. 7
4.4 Style and content module analysis
- Localized style encoding raises style-aware performance from 33.6% to 72.8%, compared with 52.8% for universal style encoding using component-wise features.The localized representation reorganizes component-wise style features; the universal alternative averages the reference component-wise styles.Source: §4.4 · p. 7, Table 3 · p. 8
- Because characters with identical components can differ in component locations, LF-Font uses a content encoder to capture structural information.The analysis compares LF-Font with this encoder removed and with DM-Font's persistent-memory content representation.Source: §4.4 · pp. 7–8
- Adding the content encoder raises few-shot harmonic mean accuracy from 21.7% to 83.2%; DM-Font's persistent memory raises content performance from 15.4% to 53.0% but worsens stylization.The authors attribute DM-Font's stylization cost to its source-style replacement strategy for unseen components.Source: §4.4 · p. 8, Table 4 · p. 8
- In one-shot generation, a reference glyph too simple to yield solid component-wise features can produce poor visual quality, making reference selection consequential.The authors note that references with richer local detail are advantageous for high-quality generation.Source: §4.4 · p. 8
5 Conclusion
LFFont combines character-wise local style representations with factorization to reconstruct unseen character-wise representations from few references, and outperforms state-of-the-art methods in multiple metrics.
5 Conclusion
- LFFont uses character-wise style representations and style/component factorization to preserve local details and outperform state-of-the-art few-shot methods, particularly on style-aware benchmarks.The authors report high-quality samples with few references, such as eight.Source: §5 · p. 8
A.1 Reference image samples
The figure shows the eight reference samples per style used for Figure 4, arranged to match its columns.
A.1 Reference image samples
- Each row corresponds to two columns of Figure 4 in the same order, showing the eight reference samples per style.The caption specifies that the reference samples are those used in Figure 4.Source: Figure 7 · p. 10
A.2 Parameter search
The parameter search examines the consistency and component-classification losses and varies factor size, selecting k = 8 for the remaining experiments.
A.2 Parameter search
- 44.5 → 69.3 style-aware accuracy is reported with the component-classification loss, and the authors say both it and the consistency loss enhance overall performance.Table 5 reports combinations of the two losses; the chosen design is its bottom row, which has the best overall performance.Source: §A.2 · p. 10, Table 5 · p. 10
- k = 8 is the factor size used in the remaining experiments.The factor-size study reports results for varying k.Source: Table 6 · p. 10, §A.2 · p. 10
A.3 Reference size study
The reference-size study reports LF-Font's stronger style-aware performance, including in one-shot generation, while the accompanying analyses examine interpolation and Korean generation.
A.3 Reference size study
- LF-Font outperforms other methods in style-aware metrics across reference-set sizes and in one-shot often exceeds their many-shot results.FUNIT and AGIS-Net retain stable content-aware performance in the low-reference regime, but their outputs are less stylized than LF-Font's; most methods improve with more references, whereas DM-Font's overall Chinese performance drops.Source: §A.3 · p. 10, Figure 8 · p. 11, Figure 9 · p. 11
A.4 Style interpolation
- Style interpolation shows LF-Font varying local features such as component size, serif-ness, and thickness while holding content representation and component factors fixed.The generated glyphs in each row differ only in style factors.Source: §A.4 · p. 10, Figure 10 · p. 11
A.5 Few-shot Korean generation
- LF-Font outperforms DM-Font and AGIS-Net in overall metrics, particularly style-aware metrics, in Korean generation with four reference glyphs.The results average LPIPS, FID, and accuracies across ten runs with different reference selections.Source: §A.5 · pp. 10–11
B Implementation Details
LF-Font’s component-wise style encoder combines convolutional, residual, component-conditional, global-context, and attention modules. Component-conditional channel-wise biases represent individual components, while global-context and CBAM blocks support component capture.
B.1 Network architecture details
- The component-wise style encoder combines convolutional, residual, component-conditional, global-context, and CBAM modules.The content encoder and generator use convolutional and residual blocks.Source: §B.1 · p. 11
- Component-conditional channel-wise biases assign a bias value to each component.Global-context and CBAM blocks are used to enable the network to capture correct components.Source: §B.1 · p. 11
B.2 LF-Font implementation details
LF-Font is trained in two phases, then evaluated alongside other models in Korean few-shot generation examples and performance comparisons. The Chinese DM-Font baseline is modified to accommodate characters with varying component counts.
B.2 LF-Font implementation details
- After 800k Chinese or 200k Korean iterations without factorization modules, LF-Font adds them and jointly trains all modules for 50k iterations.The first phase uses same-style references and targets; the second uses varied-style references, and reference reconstruction helps preserve first-phase performance.Source: §B.2 · pp. 11–12
- Figure 11 compares Korean generated samples with ground-truth glyphs, using four reference images shown in the top row.Source: Figure 11 · p. 12
- Table 7 compares AGIS-Net, DM-Font, and LF-Font on Korean few-shot generation using LPIPS, FID, and accuracy averaged over 10 runs.Source: Table 7 · p. 12
B.3 DM-Font modification details
- For Chinese experiments, DM-Font’s multi-head structure becomes component-conditioned, with averaged component-wise style features input to its decoder.The modifications also replace attention blocks with CBAM and remove decoder hourglass blocks; Korean experiments use the official model and trained weights.Source: §B.3 · p. 12
B.4 Evaluation classifier details
The evaluation-classifier details passage specifies CutMix settings and AdamP training hyperparameters.
B.4 Evaluation classifier details
- CutMix probability and beta are both 0.5.Source: §B.4 · p. 12
- AdamP training uses batch size 64, learning rate 0.0002, and 20 epochs.Source: §B.4 · p. 12