ClearText-Video: A Large-Scale Text-Centric Video Dataset Bridging Video Restoration and Scene-Text Enhancement

Motivation

• Existing TextVQA benchmarks largely overlook resolution variation, leaving MLLM reading and reasoning on in-the-wild text across diverse quality conditions insufficiently examined.Input quality issues such as blur, compression artifacts, and poor resolution can reduce answer reliability.Source: §1 · pp. 2–3

Benchmark design

• CTVid contains 4.6K+ real-world Chinese/English egocentric videos, 550K+ frames, and 1.6M annotated text instances with detection boxes, transcripts, and captions.Each content instance has High-Quality, Degraded-Quality, and Restored-Quality variants for controlled analysis of quality changes.Source: Figure 1 · p. 2, §1 · p. 3

Evaluation

• Evaluation of 16 state-of-the-art MLLMs on CTVid reveals substantial headroom for improvement, especially under quality variation.The benchmark contribution also reports persistent gaps in multilingual, quality-varying, text-rich settings.Source: §1 · p. 3

Task families

• CTVid defines Text-Centric Video Restoration and Multi-Quality VideoQA, covering image/video super-resolution, deblurring, and spatial and temporal understanding.Together, the tasks connect low-level restoration with high-level reasoning to evaluate textual fidelity and robustness under degradation.Source: Figure 1 · p. 2, §1 · pp. 3–4

Real-world Video Restoration

• For imperfectly captured, blurred, or compressed footage, diffusion-based restorers improve perceptual quality but underaddress semantics and may hallucinate character details.Examples include Upscale-A-Video, MGLD-VSR, FlashVSR [79], DiffVSR [27], and DOVE.Source: §2 · pp. 4–5

Multimodal Large Language Models

• MLLMs read text in images and videos, but performance depends on dataset diversity and scale, and systematic evaluation of video-quality effects remains limited.The work describes MLLMs as well suited to text VQA and notes recent extensions to dynamic video understanding.Source: §2 · p. 5

Text-Aware VQA Benchmarks

• TextVQA, ST-VQA, and ESTVQA ground image questions in scene text, while MTVQA and EgoTextVQA broaden language coverage.Image-based settings do not capture temporal reasoning in video.Source: §2 · p. 5
• Video benchmarks cover on-screen text, spatiotemporal grounding [77], and OCR scenarios [43], while RoadTextVQA offers limited compositional-reasoning and robustness challenges.The passage names NewsVideoQA, ViTXT-GQA [77] built on M4-ViteVQA, and MME-VideoOCR [43] as examples.Source: §2 · p. 5

Video Collection

• Videos are captured with a Sony A7R V, center-cropped from 10 seconds to 4 seconds, and sampled into 120 frames for text annotation; about 15% come from online content.The variable-focal-length lens supports recording at multiple zoom levels and from diverse perspectives.Source: §3 · p. 6

Labeling and QA Generation

• Annotators refine PaddleOCR text-box candidates and transcripts, then experts verify annotations and answers, remove ambiguous questions, correct errors, and screen for sensitive content.Questions cover temporal and spatial understanding; GPT-4o proposes candidate types, while rules instantiate questions and answers from clip metadata.Source: §3 · p. 6
• For each HQ video, bicubic downscaling (×4) and local motion blur create content-matched low-resolution and blurry variants while keeping videos, annotations, and QA pairs fixed.These synthetic degradations isolate resolution and blur effects but do not cover all real-world artifacts; the procedure follows prior work.Source: §3 · p. 6

Dataset Statistics

• CTVid contains 4,639 HQ videos, 550K+ frames, 1.6M annotated text instances, and 220K+ question–answer pairs, split into 4,327 training and 312 test videos.The test set includes offline and online clips, balanced English/Chinese coverage within each source type, and easy, medium, and hard questions.Source: §3 · p. 6

Dataset Comparison

• The same videos support paired HQ/DQ/RQ evaluation, combining high-resolution video, explicit text regions, and frame-level captions to test whether restoration preserves evidence for reasoning.The dataset also offers higher resolution than the compared VideoQA datasets.Source: §3 · p. 7

Data Release and Licensing

• The release will include splits, annotations, captions, trajectories, QA pairs, degradation scripts, evaluation materials, prompts, and baseline outputs under source-appropriate licensing and redistribution conditions.Self-captured clips and annotations will use a research license; restricted online clips will be represented by permitted source information and materials, with raw-video redistribution only when allowed.Source: §3 · p. 7

4 ClearText-Video Benchmark

• CTVid’s multi-quality VideoQA uses High-Quality, Degraded-Quality, and Restored-Quality video triplets to evaluate MLLMs across spatial and temporal text understanding.The paired, temporally aligned videos also support image and video super-resolution and video deblurring.Source: §4 · p. 8

4.1 Text-Centric Video Restoration

• Restoration tasks cover image super-resolution from 4× bicubic downsampling, video super-resolution from low-resolution inputs, and deblurring from synthetically blurred inputs.Frames include captions for semantic-aware restoration; blur is synthesized by RIFE 16× interpolation from 30 fps to 480 fps, then temporal fusion of every 65 frames.Source: §4.1 · p. 8

4.2 Text-Centric Multi-Quality VideoQA

• Spatial VideoQA uses text bounding-box regions to generate questions requiring spatial relations, while temporal VideoQA asks about text visibility, motion, transitions, scale, and boundary interactions.Spatial questions are formulated by GPT-4o and manually checked to require spatial reasoning beyond text matching; temporal questions use tracked text trajectories and rule-based generation, followed by annotator screening to reduce hallucination risk and support reproducibility.Source: §4.2 · pp. 9–10

4.3 Evaluation Protocol.

• Restoration evaluation combines full- and no-reference image-quality metrics with PaddleOCR text-detection and recognition measures; VideoQA uses accuracy, uncertainty-aware accuracy, overconfidence ratio, and abstention rate.Image metrics include PSNR, SSIM, LPIPS, DISTS, FID, NIQE, MANIQA, MUSIQ, and CLIPIQA; text measures include Pdet, Rdet, F1det, Accrec, and NEDrec. UAcc and OC couple correctness with confidence, while Abs measures withheld answers.Source: §4.3 · p. 10

5.2 Evaluation on Text-Centric Video Restoration

• Strong perceptual image-SR scores do not ensure text fidelity: SwinIR and SeeSR lead perceptual metrics, while S3Diff achieves the highest text detection and recognition accuracy.Classical methods tend to over-smooth small or degraded characters, while diffusion methods may hallucinate details; examples include SeeSR misreading “Fitness” and “Wellness,” and SwinIR failing to produce legible letters.Source: §5.2 · p. 11
• 63.36% Rdet, 74.53% F1det, and 44.72% Accrec are DOVE’s best video-SR scores, while RealViFormer leads NEDrec at 46.37% and BasicVSR++ leads Pdet at 91.11%.Video-based methods can use temporal information to improve consistency and legibility, but all evaluated methods introduce some text distortion.Source: §5.2 · p. 11
• 69.99% Rdet, 78.43% F1det, and 58.93% Accrec are Stripformer’s best video-deblurring scores; ShiftNet leads Pdet at 91.05% and NEDrec at 59.56%.Several methods recover text with minor distortions, but large motion can cause partial restoration or complete breakdown when it exceeds model capacity.Source: §5.2 · p. 12

Text-Centric VideoQA for Spatial Understanding.

• 71.67%, 65.00%, and 70.00% are Gemini-2.5-pro’s spatial QA accuracies on HQ, DQ-Low_res, and RQ-S3DIFF, while other restoration conditions have different leaders.Claude-Sonnet-4.6 leads on DQ-Blur and RQ-MIMO (60.00% and 70.00%), and Gemini-2.5-flash leads on RQ-DOVE (66.67%). Qwen2.5-VL-7B-SFT achieves the best open-source accuracy across all six conditions, improving over its base model by 6.31–10.11 percentage points; the tuned model reduces overconfidence, but accuracy gains do not always improve UAcc.Source: Text-Centric VideoQA for Spatial… · pp. 12–13, Figure 6 · p. 12
• Averaged across 16 MLLMs, blur reduces spatial QA accuracy by 6.05 points from HQ, versus 3.14 points for low resolution.An OCR+LLM diagnostic baseline remains far below direct MLLM inference, indicating the task requires spatial grounding and multimodal reasoning beyond extracted OCR text.Source: Text-Centric VideoQA for Spatial… · p. 13

Text-Centric VideoQA for Temporal Understanding.

• 60.02%, 59.45%, 59.34%, 58.88%, and 60.37% are Claude-Sonnet-4.6’s accuracies on HQ, DQ-Blur, RQ-MIMO, RQ-S3DIFF, and RQ-DOVE temporal QA.Claude-Sonnet-4.6 is the only model to exceed 58% in any condition and leads the strongest open-source model by more than five points in every condition, reaching a 6.30-point margin on RQ-MIMO. Rankings vary with quality, and restoration can produce unstable changes.Source: Text-Centric VideoQA for Temporal… · p. 13

Restoration and understanding

• Restored videos can become sharper yet alter character strokes, introduce plausible text, or shift spatial cues, making some MLLMs less accurate than on degraded inputs.The authors argue that restoration should be evaluated with both visual/text-fidelity metrics and downstream reasoning metrics; CTVid uses restoration to test this relationship, which prior single-quality TextVQA benchmarks cannot isolate.Source: §6 · pp. 13–14

Controlled degradations and data release

• Controlled bicubic downsampling and motion blur keep content, annotations, and QA pairs fixed across HQ/DQ/RQ comparisons, but omit several real-world artifacts.The omitted conditions include exposure changes, rolling shutter, sensor noise, ISP effects, and complex compression; future versions may extend the paired protocol, and the release includes annotations, QA pairs, code, prompts, permitted video assets, and metadata.Source: §6 · p. 14

OCR probes and calibration

• PaddleOCR initializes coarse annotation text candidates and provides a fixed fidelity probe, while human annotators verify boxes, transcripts, QA pairs, and ambiguity.The OCR+LLM diagnostic baseline shows that extracted text alone is insufficient because CTVid also requires spatial grounding and multimodal reasoning.Source: §6 · p. 14
• Supervised fine-tuning on CTVid improves answer accuracy but does not uniformly improve reliability, with lower UAcc and changed overconfidence indicating an accuracy-calibration trade-off.The paper identifies explicit reliability calibration as a future modeling need.Source: §6 · p. 14

7 Conclusion

• CTVid evaluates the same text-rich videos in High-Quality, Degraded-Quality, and Restored-Quality regimes, linking low-level restoration with high-level video understanding.Its suite covers image and video super-resolution, video deblurring, and multi-quality VideoQA, with human-verified text annotations, captions, and spatial/temporal QA pairs.Source: §7 · pp. 14–15
• CTVid is presented as the first large-scale, scene-text-aware video QA benchmark designed to examine how input video quality affects text-centric multimodal reasoning.The authors expect it to support research on text-faithful restoration, quality-robust multimodal reasoning, and evaluation that jointly measures visual enhancement and text-grounded understanding.Source: §7 · pp. 14–15
• Experiments with 18 restoration methods and 16 MLLMs show current systems remain fragile under quality changes: restoration does not always improve reasoning accuracy and can alter textual evidence.The benchmark evaluates whether visual enhancement preserves the text needed for downstream reasoning.Source: §7 · p. 15

A.1 Diversity of Text-Centric Video Scenes

• ClearText-Video covers transportation hubs, urban driving, commercial areas, and public facilities, where text conveys navigation, transportation, and commercial guidance.Source: §A.1 · p. 21
• English and Chinese scenes include layouts from large billboards and vehicle liveries to dense information boards, with text viewed at varied distances, viewpoints, and clutter levels.Source: §A.1 · p. 21
• Fine-grained bounding boxes localize text instances over time, supporting evaluation across languages, scene categories, and visual conditions rather than one dataset’s biases.Source: §A.1 · p. 21
• The scene diversity supports both video text spotting and reading and higher-level video-language understanding in realistic, text-rich settings.Source: §A.1 · p. 21

A.2 Test Set Details

• The test set contains 312 well-lit, text-clear HQ videos split equally between English and Chinese, excluding clips with excessive global motion blur.Source: §A.2 · p. 21
• Matched ×4 bicubic-downscaled LR and locally blurred versions of each HQ test video enable controlled evaluation across HQ, LR, and blur with identical content and question distributions.Source: §A.2 · p. 21
• The test set is 74% offline and 26% online; offline clips mix artificial and natural lighting, while online clips are almost entirely artificially lit.Source: §A.2 · pp. 21–22, Figure 7 · p. 22

A.3 Release, Licensing, and Dataset Comparison

• CTVid will release annotations, QA data, scripts, evaluation resources, and baseline outputs, with distribution of source videos governed by license and permission constraints.Self-captured clips and annotations use a research license; restricted online clips may be represented by source IDs or URLs, clip ranges, annotations, and preprocessing scripts.Source: §A.3 · p. 22

A.4 Question Answering Generation Details

• Spatial QA combines box-grounded text questions with rule-based region selection and LLM phrasing, while temporal QA uses text trajectories and human screening.Spatial items include multiple-choice, true/false, and fill-in-the-blank questions; temporal generators cover visibility, localization, motion, scale, and boundary interaction.Source: §A.4 · pp. 23–25

B Additional Experimental Details

• The appendix details image/video super-resolution and deblurring baselines, then describes the MLLMs used in spatial and temporal VideoQA experiments and qualitative analyses.The final quantitative sets comprise 16 spatial MLLMs and an eight-model temporal subset; Grok-4-fast is used only for temporal qualitative visualizations.Source: B Additional Experimental Details · p. 25, §B.4 · p. 30

B.1 Details of Image Super-Resolution Models

• Image-SR baselines include blind real-world, transformer, and diffusion methods, including Real-ESRGAN [52], SwinIR, SeeSR, OSEDiff, S3Diff [68], and AdcSR.Their designs include synthetic degradation modeling, shifted-window attention, semantic prompts, and one-step diffusion approaches.Source: §B.1 · pp. 26–27

B.2 Details of Video Super-Resolution Models

• Video-SR baselines include RealBasicVSR, BasicVSR++ [5], RealViFormer, MGLD-VSR, Upscale-A-Video, and DOVE.Their methods use frame cleaning and propagation, second-order alignment, attention-based fusion, motion-guided diffusion, or recurrent latent refinement for temporal consistency.Source: §B.2 · pp. 27–28

B.3 Details of Deblurring Models

• Deblurring baselines cover image and video models, including MIMO-UNet+ [11], NAFNet [8], Restormer, Stripformer, RVRT [29], and ShiftNet.Configurations use GoPro, RealBlur-J/RealBlur-R, or DVD data, spanning single-image architectures and recurrent or spatiotemporal video restoration.Source: §B.3 · pp. 28–30

B.4 Details of MLLMs

• The spatial evaluation includes 16 MLLMs and the temporal table an eight-model subset; Qwen2.5-VL-7B-SFT is spatial-only, while Grok-4-fast is qualitative-only.Kimi-VL-16B [22] is described as the highest-scoring open-source baseline under most temporal quality conditions.Source: §B.4 · pp. 30, 32

• Restoration can introduce spurious or distorted character strokes, while MLLMs can fail through localization bias, reasoning errors, or temporal aggregation mistakes.These failure modes help explain why sharper restored videos do not always improve downstream QA.Source: §B.5 · p. 32, Figure 11 · p. 33
• Across 16 MLLMs, LR reduces spatial QA accuracy by 3.14 points from HQ, while blur reduces it by 6.05 points.The same trend appears for Qwen3-VL-8B.Source: §B.5 · p. 32, Table 7 · p. 33
• OCR+LLM remains far below direct Qwen2.5-VL-7B; RQ improves direct Qwen2.5-VL-7B over DQ but does not improve OCR+LLM.PaddleOCR extracts text from each quality version, and Qwen2.5-7B answers from OCR text alone; HQ OCR uses original HQ inputs, not oracle human transcripts.Source: §B.5 · pp. 32–33

Video deblurring

• Stripformer produces relatively clear deblurred outputs but changes the COSTCO sign’s color and alters “Jingu” to “Jimgu,” while MIMO-UNet+, Restormer, and RVRT provide limited deblurring.Source: §B.6 · p. 33

Image super-resolution

• Image super-resolution works well on larger English text but can alter it, render dense Chinese characters illegibly, or produce unrelated text such as “Paln Sharo” for “Ride Share.”SeeSR renders “South” as “Sauth,” and all evaluated methods struggle with dense Chinese characters and smaller text in the examples.Source: §B.6 · p. 34

Video super-resolution

• Video super-resolution appears to preserve text more accurately, but Upscale-A-Video changes individual letters and DOVE slightly deforms “COSTCO” and omits strokes in Chinese text.Examples include “Commarcial” for “Commercial” and “Lθading” for “Loading.”Source: §B.6 · p. 34

Overall findings

• Video-based methods often preserve text more accurately than image-only methods, but they still struggle with compact Chinese characters and may introduce text distortions that harm downstream performance.The examples motivate greater use of temporal priors in future diffusion-based video restoration research.Source: §B.6 · p. 34

B.7 Spatial VideoQA Visualization under Multi-Quality Videos

• Each example pairs a localized text region and spatial question with a keyframe, ground-truth answer, and predictions from 16 MLLMs across six quality conditions.The conditions are HQ, DQ-Low_res, DQ-Blur, RQ-DOVE, RQ-MIMO, and RQ-S3DIFF; questions include reading street names or signs and completing missing characters within bounding boxes.Source: §B.7 · p. 34
• Models often miss tokens, ordering, similar answer options, or complete fill-in-the-blank answers; degradation weakens spatial grounding, and restoration does not ensure correct reasoning.Several models appear to rely excessively on partial OCR cues, even when restored text looks sharper; the examples clarify quantitative trends in Table 4.Source: §B.7 · pp. 34–35
• Qwen2.5-VL-7B-SFT shows more stable predictions across quality conditions, consistent with its greater robustness in numerical evaluation.Source: §B.7 · p. 35

B.8 Temporal VideoQA Visualization under Multi-Quality Videos

• Across quality conditions, models give inconsistent temporal estimates and often rely on static cues instead of aggregating visibility, motion, and duration evidence across the full sequence.The examples compare HQ, DQ-Blur, RQ-DOVE, RQ-MIMO, and RQ-S3DIFF videos using eight temporal models; even comparatively robust proprietary models fluctuate across conditions.Source: §B.8 · pp. 35–37
• On ClearText-Video’s text-centric temporal VideoQA example, most models predicted the text would stay roughly in place despite ground-truth motion consistently leftward under all quality conditions; only a few predicted correctly.Their predictions still became unstable under degraded or restored inputs.Source: Figure 16 · p. 39
• Many models confuse the text’s dominant bottom-left position with the bottom-right or center, showing that location must be aggregated across the sequence.Some predictions also vary across quality conditions.Source: Figure 17 · p. 39
• Most models incorrectly judge whether “PHARMACY” touches or runs off the frame edge, often labeling it fully inside or merely near the boundary.These errors are especially evident under degraded and restored inputs.Source: Figure 18 · p. 40