PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Pixel Reconstruction: Retrieval-Guided Glyph-Aware Restoration for Low-Resource Manchu Historical Documents

Ting Huang, Dongdong Wang, Mingqiu Liang, Siyang Lu

arXiv:2610.00315v1cs.CVcs.AIcs.LG

TL;DR

Low-resource historical restoration must recover glyph structures that pixel-focused methods may not preserve, especially amid limited data and similar glyphs. CRGR retrieves and verifies glyph references, and experiments report improved glyph fidelity while preserving intact document contents. Its failures remain bounded by reference coverage and available structural evidence.

  • Problem

    Pixel-centric restoration can yield visually plausible but structurally incorrect glyphs, while low-resource scripts often have limited data and similar glyph forms.

    Source: §1 · p. 1

  • Method

    CRGR retrieves structurally similar glyph candidates, verifies them with a frozen VLM, and restores using the selected reference without task-specific fine-tuning.

    Source: §1 · pp. 1–2

  • Results

    Experiments report that CRGR achieves superior glyph fidelity and document content preservation compared with representative restoration methods, and improves glyph restoration fidelity while preserving intact contents.

    Source: §1 · p. 2, §5 · p. 7

  • Takeaways & Limitations

    The paper concludes that low-resource historical restoration should move beyond unconstrained pixel generation toward retrieval-guided semantic restoration.

    Source: §5 · p. 7

    CRGR is limited by reference-gallery coverage and discriminative structural evidence; severe degradation or ambiguous glyph identities can make retrieval and verification uncertain.

    Source: §4.4 · p. 7

Abstract

from arXiv · show

Historical Manchu documents preserve invaluable linguistic and cultural heritage, yet their digitization is hindered by severe degradations and the scarcity of paired training data. Existing document restoration methods primarily optimize pixel-level reconstruction, which can produce visually plausible results while failing to preserve the structural identity of Manchu glyphs. To address this limitation, we propose a retrieval-guided glyph-aware restoration framework that goes beyond pixel reconstruction by explicitly incorporating glyph-level structural knowledge. Our method retrieves relevant glyph exemplars to provide structural guidance during restoration and integrates this information into the reconstruction process, improving the recovery of degraded character structures under low-resource conditions. Extensive experiments on Manchu historical documents demonstrate that the proposed approach improves both image restoration quality and glyph-level fidelity compared with existing restoration methods. These results highlight the importance of incorporating character-aware structural priors for reliable restoration of low-resource historical documents.

1 Introduction

Low-resource Manchu restoration is difficult because historical collections are limited, glyph styles vary, and degradation can erase distinguishing structure. CRGR reframes restoration as retrieval-guided glyph recovery, combining coarse retrieval and VLM verification to select references rather than synthesize glyphs freely.

1 Introduction

  • Manchu and other under-resourced scripts have limited historical collections, varied glyph styles, and complex degradation that makes accurate glyph-structure recovery challenging.Degradation includes aging, insect damage, ink fading, and physical deterioration.Source: §1 · p. 1
  • Pixel-centric restoration can produce visually plausible results with incorrect glyph structures, risking errors in OCR, historical interpretation, and linguistic analysis.For low-resource scripts with similar glyphs, visual similarity is not a reliable proxy for glyph correctness.Source: §1 · p. 1
  • CRGR retrieves structurally similar glyph candidates, verifies them with a VLM, and uses the selected reference for restoration instead of open-ended pixel synthesis.Its coarse-to-fine process progressively reduces candidate ambiguity.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • CRGR achieves superior glyph fidelity and document content preservation compared with representative restoration methods on a Manchu historical document dataset.The paper presents this result as evidence for its retrieval-guided restoration paradigm.Source: §1 · p. 2

2 Related Work

Related work moves from visual reconstruction toward explicit structural constraints and retrieval or vision-language guidance. The review identifies unresolved challenges in distinguishing similar glyphs and in applying retrieval-guided semantic restoration to low-resource documents.

2.1 Visual Reconstruction

  • Visual-reconstruction methods use image context to restore plausible content, but appearance consistency may not ensure structural correctness for visually similar low-resource glyphs.The approaches reviewed include inpainting, large-receptive-field, diffusion-based, and document-oriented restoration methods.Source: §2.1 · p. 2

2.2 Structure-aware Restoration

  • Structure-aware methods model strokes, character-component relationships, glyph-generation constraints, adversarial consistency, or reference exemplars, yet visual similarity still limits discrimination among similar glyphs.The review notes that preserving glyph structure alone does not resolve the challenge of distinguishing visually similar patterns.Source: §2.2 · pp. 2–3

2.3 Retrieval-guided and Vision-language Restoration

  • Retrieval and vision-language approaches improve glyph discrimination, but many rely on abundant annotations, task-specific adaptation, or end-to-end multimodal optimization.The review states that retrieval-guided semantic restoration using authentic glyph references remains insufficiently explored for low-resource historical documents.Source: §2.3 · p. 3

3 Methodology

CRGR restores glyphs by retrieving authentic historical references, verifying candidates, and replacing damaged glyphs; its evaluation protocol measures glyph identity and structural fidelity separately from image fidelity.

3.1 Framework Overview

  • CRGR retrieves structurally similar glyphs, verifies candidates with a frozen VLM, then inserts the selected reference into the corrupted region.The two stages are coarse glyph retrieval followed by fine-grained candidate verification.Source: §3.1 · p. 3

3.2 Glyph Knowledge Base

  • The fixed inference-time gallery contains authentic glyphs cropped from intact historical documents, constraining restoration to reference forms rather than synthesized pixels.A pretrained detector localizes and crops the intact glyphs to form the gallery.Source: §3.2 · p. 3

3.3 Coarse Glyph Retrieval

  • A vision encoder ranks gallery glyphs by similarity and retains Top-K candidates for verification rather than selecting the highest-ranked result directly.Keeping candidates for verification reduces the search space while ensuring restored glyphs originate from authentic historical references.Source: §3.3 · pp. 3–4

3.4 VLM-based Candidate Verification

  • A frozen VLM scores the damaged glyph against each retrieved candidate, and the highest-scoring candidate becomes the reference for aligned replacement.The selected reference is aligned to the target position to complete restoration.Source: §3.4 · p. 4

3.5 Glyph-oriented Evaluation Protocol

  • The protocol uses glyph detection and representation extraction, then evaluates identity preservation and structural fidelity with GRA, CCR, and GSS.GRA measures correct glyph identities; CCR measures confusion between visually similar categories; GSS assesses structural fidelity using deep glyph features.Source: §3.5 · p. 4
  • By separating glyph identity correctness from structural similarity, the detector-agnostic protocol assesses semantic glyph-level restoration beyond conventional image-level metrics.Its localization and representation components are modular and can incorporate future detector and feature-extraction advances.Source: §3.5 · p. 4

4 Experiments

Experiments evaluate CRGR on a constructed Manchu–Han benchmark and report glyph-oriented gains, while ablations and failure cases expose dependencies on retrieval quality and reference evidence.

4.1 Experimental Setup

  • The study constructs a Manchu–Han woodblock benchmark with over 15,000 gallery glyphs and evaluates full-page and 3×3 patch-level restoration against five representative methods.The benchmark covers scanned documents from multiple sources; patches containing masked glyphs are retained.Source: Dataset · p. 4, Baselines · p. 4

4.2 Comparison with Representative Restoration Methods

  • CRGR restores damaged glyphs from verified gallery references rather than synthesizing missing pixels, preserving glyph structures, printed stroke characteristics, and document layout in qualitative comparisons.LaMa and AOT-GAN may produce masked-region artifacts, while diffusion methods can yield glyph appearances that differ from original structures.Source: Qualitative Comparison · p. 5
  • PixelHacker slightly outperformed CRGR on masked GRA and CCR, but CRGR's stronger structural similarity shows that category-level correctness need not ensure full glyph preservation.CRGR also achieved the lowest LPIPS score among the methods, although pixel-level metrics do not fully reflect glyph identity or structural consistency.Source: Quantitative Comparison · pp. 5–6
  • On masked regions, CRGR achieved the highest GSS (0.9986) and best GSSm and GSSh; on intact regions, it led GRA (0.9600) and CCR (0.0377), with competitive GSS.The comparison reports glyph-oriented metrics for both masked and intact regions.Source: Quantitative Comparison · pp. 5–6, Table 2 · p. 6

4.3 Ablation Study

4.4 Failure Analysis

  • Failures arise when the gallery lacks the correct glyph, degradation removes discriminative cues, or visually similar glyphs remain ambiguous during verification.These cases limit CRGR through reference coverage and available structural evidence.Source: Figure 4 · p. 7, §4.4 · p. 7

4.3 Ablation Study

  • At K = 5, CRGR achieves the highest GRA of 0.6610 with competitive structural consistency; larger candidate pools do not consistently improve overall performance and increase verification cost.Small K risks excluding the ground-truth glyph, whereas excessive candidates increase ambiguity and VLM discrimination complexity.Source: Effect of Candidate Number · pp. 6–7, Table 5 · p. 6

5 Conclusion

CRGR replaces unconstrained pixel generation with coarse-to-fine glyph retrieval and VLM-based candidate verification. Experiments report improved glyph restoration fidelity while preserving intact document contents.

  • CRGR performs coarse-to-fine glyph retrieval followed by VLM-based candidate verification, using authentic historical references rather than synthesized glyphs.Source: §5 · p. 7
  • Experiments show improved glyph restoration fidelity while preserving intact document contents.Source: §5 · p. 7
  • The study suggests retrieval-guided semantic restoration, combining authentic references and multimodal reasoning, for structural preservation in low-resource historical documents.Source: §5 · p. 7

Related papers

Loading 2610.00315v1…