PRISM: Prior Rectification and Uncertainty-Aware Structure Modeling for Diffusion-Based Text Image Super-Resolution

Zihang Xu; Xiaoyang Liu; Zheng Chen; Yulun Zhang; Xiaokang Yang

Source-linked AI summary

PRISM: Prior Rectification and Uncertainty-Aware Structure Modeling for Diffusion-Based Text Image Super-Resolution

Zihang Xu, Xiaoyang Liu, Zheng Chen, Yulun Zhang, Xiaokang Yang

arXiv:2605.13027v1May 13, 2026cs.CV

TL;DR

Text-SR must preserve character identity despite unreliable text conditions and underdetermined local stroke boundaries in severe degradation. PRISM uses FMPR to rectify global priors and SURE to refine uncertain structure within a single diffusion pass. It reports state-of-the-art performance with millisecond-level inference on synthetic and real-world benchmarks.

Problem
Severe degradation can make extracted text conditions unreliable, while plausible global priors do not fully determine fine-grained stroke boundaries needed for faithful character restoration.
Method
PRISM combines FMPR, which transports degraded embeddings toward a privileged prior learned from paired LQ/HQ latents, with SURE, which uncertainty-gates local structural residuals.
Results
State-of-the-art performance with millisecond-level inference is reported on synthetic and real-world benchmarks.
Takeaways & Limitations
PRISM explicitly separates global prior rectification from local structure refinement while retaining one-step diffusion restoration efficiency.
Takeaways & Limitations
The study focuses on Chinese-English text-line super-resolution with moderate to long aspect ratios and limited language coverage, excluding full-scene, dense multi-line, and highly irregular layouts.

Abstract

from arXiv · show

Text image super-resolution (Text-SR) requires more than visually plausible detail synthesis: slight errors in stroke topology may alter character identity and break readability. Existing methods improve text fidelity with stronger recognition-based or generative priors, yet they still face two unresolved challenges under severe degradation: the text condition extracted from low-quality inputs can itself be unreliable, and a plausible global prior does not fully determine fine-grained stroke boundaries. We present PRISM, a single-step diffusion-based Text-SR framework that addresses these two challenges through Flow-Matching Prior Rectification (FMPR) and a Structure-guided Uncertainty-aware Residual Encoder (SURE). FMPR constructs a privileged training-time prior from paired low-quality/high-quality latents and learns a flow matching that transports degraded embeddings toward this restoration-oriented prior space, yielding more accurate and reliable global text guidance. SURE further predicts uncertainty-aware structural residuals to selectively absorb reliable local boundary evidence while suppressing ambiguous stroke cues. Together, these components enable explicit global prior rectification and local structure refinement within a single diffusion restoration pass. Experiments on both synthetic and real-world benchmarks show that PRISM achieves state-of-the-art performance with millisecond-level inference. Our dataset and code will be available at https://github.com/faithxuz/PRISM.

1 Introduction

Text-SR must preserve semantically faithful glyph structures because small stroke errors can change character identity. PRISM addresses unreliable global text conditions and ambiguous local boundaries through FMPR, SURE, and one-step diffusion restoration.

Motivation: Text-SR must recover visually plausible details and semantically faithful glyph structures with sub-character precision.Broken strokes, merged components, or distorted enclosures can change character identity, especially in densely structured scripts.
Challenges: Severe degradation makes inferred text conditions unreliable, while plausible global semantics cannot fully determine pixel-aligned stroke closures and intersections.Entangled condition estimation and reconstruction can produce sharp but semantically incorrect outputs.
Method: FMPR transports degraded embeddings toward a privileged restoration-oriented prior space learned from paired LQ/HQ latents.The flow directly models the velocity field toward restoration-oriented text tokens for more reliable global guidance.
Method: SURE predicts structural-feature means and uncertainty, selectively absorbing reliable boundaries while suppressing ambiguous stroke cues.This avoids treating low-quality edges as deterministic truth during local stroke-geometry refinement.
Results: PRISM achieves state-of-the-art performance on synthetic and real-world benchmarks with millisecond-level inference.The compact FMPR transport and single diffusion-backbone call preserve one-step restoration efficiency.
Contributions: The framework combines privileged-prior rectification, uncertainty-aware structure guidance, and single-step diffusion inference.These contributions target prior reliability and structural uncertainty under severe degradation.

2 Related Works

Related work improves Text-SR and broader real-world restoration through recognition, text-aware, generative, and diffusion-based guidance. Text-specific methods focus on preserving character identity alongside visual quality.

Real-World Image Super-Resolution: Real-world image super-resolution addresses complex unknown degradations using degradation modeling, discriminative reconstruction, and diffusion priors.Efficient methods also target the high cost of iterative diffusion sampling.
Text Image Super-Resolution: Text-SR restores readable text crops or text-line images while preserving character identity as well as visual quality.Prior approaches use recognition guidance, sequential reasoning, layout modeling, and text-prior attention.
Text-Aware Restoration: Broader text-aware restoration introduces text-region perception, segmentation, text spotting, or text-aware conditioning into general restoration frameworks.These methods address real-world or full-image settings rather than only text crops or text lines.

3 Methodology

PRISM separates Text-SR into global prior rectification and local structure refinement. FMPR recovers a restoration-oriented text condition, while SURE models uncertainty to refine reliable stroke boundaries within a single-step diffusion pipeline.

Overall Structure: PRISM uses a pre-trained latent diffusion model to progressively address unreliable text conditions and ambiguous stroke topology.The pipeline encodes the degraded image, recovers a text-aware prior, predicts structural residuals, and decodes the restored latent.
FMPR: Flow-Matching Prior Rectification: FMPR constructs a privileged conditional prior from paired LQ/HQ latents and learns to recover it from degraded inputs alone.The privileged prior defines the target space during training, while an LQ-only pathway approximates it at inference.
FMPR: Flow-Matching Prior Rectification: The recovered prior ˆc guides restoration after flow matching transports the observed LQ embedding toward the privileged prior space.Recovery starts from the LQ-only prior and obtains ˆc after K Euler steps; the recovered condition is then used for restoration.
SURE: Structure-guided Uncertainty-aware Residual Encoder: SURE predicts a stochastic structural cue from feature-map mean and uncertainty estimates, representing ambiguous local evidence with higher uncertainty.Reparameterized cues are projected into the structure-control space and decoded into an auxiliary boundary map.
SURE: Structure-guided Uncertainty-aware Residual Encoder: SURE injects uncertainty-aware residual signals into frozen UNet skip features to refine local stroke topology, boundary closure, and spatial alignment.Reliable boundaries are emphasized, while high-uncertainty regions remain correspondingly less decisive in the structural representation.

4 Experiments

PRISM is evaluated on curated synthetic and real-world text-SR benchmarks against representative restoration methods, with quantitative, qualitative, ablation, and efficiency analyses. Results show stronger perceptual and text fidelity under severe degradation, while FMPR and SURE analyses examine prior recovery, Euler-step trade-offs, and uncertainty-aware structure guidance.

Experimental Setup: The evaluation uses BTL-test and RealCE-val for ×2 and ×4 super-resolution, with PSNR, LPIPS, FID, ACC, and NED covering fidelity, realism, and text recognition.BTL contains curated bilingual text-line pairs, while RealCE-val contains 1,037 valid real-world testing pairs.
Main Results: On BTL-test, PRISM achieves the best LPIPS, FID, and NED at both scales, plus the highest ACC under ×4.The reported advantages concern perceptual quality and text fidelity, despite a PSNR gap attributed to the perception-distortion tradeoff.
Main Results: Under RealCE-val ×4, PRISM ranks first across all metrics, improving ACC from 60.62% to 65.19% and reducing FID from 74.52 to 47.83 versus second-best results.Under ×2, it achieves the best PSNR and FID and ranks second on the remaining metrics.
Main Results: Qualitatively, PRISM better preserves character readability and local stroke continuity while maintaining more consistent backgrounds than competing methods.DiffTSR can produce broken or merged strokes, while TeReDiff may introduce false colors or redundant and incorrect strokes.
Inference Efficiency: PRISM performs final restoration in one denoising step, taking 0.08 s per 128×512 image versus 10.70 s for DiffTSR and 5.27 s for TeReDiff.FMPR itself uses 16-step Euler discretization, while the final diffusion restoration remains single-step.
Ablation Studies: FMPR ablations indicate that flow matching more effectively rectifies degraded priors than direct regression or diffusion-based prior construction.The privileged paired latent condition provides an upper bound, while flow matching transports observed priors toward the privileged prior space.
Ablation Studies: Increasing FMPR Euler steps improves realism and text fidelity before saturation; K = 16 is selected because K = 32 slightly improves results but doubles computational cost.SURE ablations likewise show deterministic edge guidance helps, while uncertainty-aware guidance suppresses unreliable stroke evidence.

5 Conclusion

PRISM addresses unreliable text-aware priors and uncertain local stroke structures under severe degradation by separating global prior rectification from local structure refinement. FMPR and SURE preserve one-step diffusion efficiency while improving character fidelity and perceptual quality across synthetic and real-world benchmarks.

Conclusion: PRISM targets unreliable text-aware prior estimation and uncertain local stroke structures in severely degraded text images.Its restoration process explicitly separates global prior rectification from local structure refinement.
Conclusion: FMPR learns a flow-matching recovery of reliable text-aware conditions from degraded inputs using a privileged prior space formed from paired LQ/HQ latents.SURE injects uncertainty-aware structural residuals to refine ambiguous stroke boundaries without overcommitting to unreliable edges.
Conclusion: Experiments on synthetic and real-world benchmarks report superior performance over representative Text-SR and text-aware restoration methods, especially under severe degradation and complex glyph structures.The framework retains one-step diffusion restoration while improving character fidelity and perceptual quality.

A Details of BTL Dataset Construction

BTL combines quality-controlled real text crops with rendered synthetic text lines to support Chinese-English Text-SR. Its mixed composition provides a better balance between sharpness and real-world appearance than either source alone.

Dataset motivation: BTL addresses limited bilingual training coverage by combining Chinese CTR crops, English SA-Text crops, and digit samples from both sources.The source pools differ in language composition, motivating source-specific allocation.
Real-image curation: Candidate crops are standardized to height 128, aspect ratios between 2 and 8, and transcripts of at most 24 characters before quality filtering.The selection protocol removes unsuitable text-line geometries and ranks candidates using no-reference image-quality assessment.
Real-image curation: The curated real subset contains 50K images, including 31,567 Chinese, 16,634 English, and 1,084 digit-only CTR or 715 digit-only SA-Text samples.Selection quotas preserve the retained candidate-group distribution.
Dataset composition: BTL adds 50K rendered synthetic HQ text-line images, balancing controllable transcripts, fonts, and layouts with real image statistics and background-text interactions.Each HQ image receives a corresponding LQ input synthesized with BSRGAN- and Real-ESRGAN-based degradation pipelines, producing 100K total images.
Data-composition analysis: BTL-train combines curated real crops and rendered text lines, yielding sharper and more naturally integrated results on RealCE-val than Synth-train or CTR-train alone.Synth-train can separate strokes from the background, whereas CTR-train tends to produce conservative and less clear text.

B Inference Speed Analysis

The inference-speed analysis compares methods on standardized RealCE-val inputs under ×4 super-resolution, including sampling steps and per-image runtime.

Speed comparison: Table 6 compares inference speed on RealCE-val under ×4 super-resolution using standardized image dimensions and reports sampling steps alongside per-image inference time.The comparison uses selected images resized to target resolution; non-diffusion methods use fixed 32 × 128 LR and 128-wide HR sizes.

C More Visualizations

PRISM is qualitatively compared with prior methods on synthetic and real-world text-line examples, while the data-composition study examines how training sources affect RealCE-val outputs.

Visualization scope: Visual comparisons cover Chinese, English, and digit-only text lines of varied lengths on BTL-test and RealCE-val.The figures include both synthetic and real-world ×4 super-resolution examples.
Comparison: Earlier methods often smooth complex Chinese characters, while generative methods can introduce distorted glyphs, missing or extra strokes, and texture artifacts.These failure patterns are reported across TSRN, TBSRN, TATT, MARCONet, DiffTSR, and TeReDiff.
Comparison: PRISM restores clearer and more coherent text structures across synthetic and real-world examples, with fewer broken or merged strokes.This comparison is presented in the visual examples from Figs. 10 and 11.
Efficiency context: Table 6 identifies the diffusion-step contrast used in the speed comparison: DiffTSR uses 200 steps, TeReDiff uses 50, and PRISM uses one.The table concerns ×4 inference on RealCE-val.

D Architecture Details of PRISM

PRISM separates prior recovery from local structural control, using token-based FMPR guidance and uncertainty-aware SURE residuals within the diffusion backbone.

FMPR prior encoders: The privileged and LQ-only prior encoders share an architecture but process different latent inputs: concatenated LQ-HQ latents versus LQ latents alone.The privileged encoder uses 8 input channels, while the LQ-only encoder uses 4.
FMPR prior encoders: The prior encoder projects latent features into 64 spatial tokens with 1024 channels for prior embedding.Adaptive pooling produces the fixed 4×16 spatial token layout.
FMPR flow matching: FMPR’s velocity network is a token-wise MLP that combines the current prior embedding with a normalized integration step and performs 16-step Euler integration.Four residual MLP blocks refine the representation before predicting the velocity tensor.
SURE spatial cues: SURE extracts multi-scale spatial features, predicts structural-cue mean and log-variance, and samples a stochastic cue for residual control.An auxiliary boundary map provides structure supervision, while an FPN produces the feature map used by the uncertainty-aware latent head.
SURE residual control: The structural residual encoder takes the degraded latent, recovered prior, and projected structural cue, then injects nine residual controls into frozen UNet skip features.This branch focuses on spatial refinement rather than re-estimating the text-aware prior.

E Broader Impacts and Limitations

PRISM targets improved readability and visual quality for degraded text images, while recognizing that reconstructed content may be unreliable in sensitive applications and that its evaluation scope remains limited.

Broader Impacts: PRISM may benefit document enhancement, scene text recognition, assistive reading, and OCR preprocessing.
Broader Impacts: Restored results should be used cautiously in legal, medical, financial, or privacy-related settings because ambiguous inputs may produce plausible content.
Limitations: The study focuses on Chinese-English text-line super-resolution with moderate-to-long aspect ratios, excluding full-scene text, dense multi-line documents, and highly irregular layouts.
Limitations: BTL's language coverage is mainly limited to Chinese, English, and digit-based text, leaving broader languages, scripts, fonts, and capture conditions for future expansion.