Enhanced Video Text Editing with Trajectory-Aligned Glyph Rendering

• Edited text must be correct in every frame and follow the scene’s position and perspective without flickering, while the rest of the video remains unchanged.Source: §1 · p. 1
• General video editors lack glyph-level control, image text editors do not model frame relations, and available video text methods have pipeline or language limits; real-scene benchmarks are also lacking.The cited examples include STRIVE’s reference-frame propagation and SteerVTE’s English-only support.Source: §1 · p. 1
• Explicit target glyphs can guide exact strokes while the generative model adapts their appearance, and recognizer-based supervision targets stroke errors that contribute little to diffusion loss.Source: §1 · p. 2
• The method formulates editing as text-conditioned inpainting along per-frame text boxes, using warped glyph references and normalized multi-depth features from a frozen recognizer.Because the original text is hidden, a real video can serve as its own ground truth for training without paired videos.Source: §1 · p. 2
• Sen.ACC rises from 0.5980 to 0.9408 over Seedance, with 64.3% of user votes versus 30.8%, while the method leads in text accuracy and background preservation.Source: §1 · p. 2
• VTEdit is a real-scene benchmark for text replacement and addition, built from videos annotated with text trajectories and comprising 440 annotated trajectories.Source: §1 · p. 2

Video Editing and Visual Text Editing

• General video editors lack glyph-level control, while framewise image text editing misses inter-frame relations; STRIVE propagates a frontalized frame, and SteerVTE supports only English (G et al., 2021; Zeng et al., 2026).Source: §2 · pp. 2–3

Recognizer-based Text Supervision

• AnyText compares single-depth recognizer features, whereas JoyType, CharGen, and SteerVTE use multilayer losses; normalizing only by feature size can let larger activations dominate (Tuo et al., 2024b; Li et al., 2024; Ma et al., 2024; Zeng et al., 2026).This method instead normalizes each depth’s error by ground-truth feature magnitude.Source: §2 · p. 3

3.1 PROBLEM DEFINITION

• Video text editing regenerates content inside per-frame text boxes while preserving the outside, using the original text in real videos as training ground truth.At inference, the target text is replaced with the desired text.Source: §3.1 · p. 3

3.2 TRAJECTORY-ALIGNED GLYPH RENDERING REFERENCE

• The trajectory-aligned reference warps rendered target glyphs into each frame’s text box, giving the model explicit guidance that follows text position and perspective.The reference guides glyph structure rather than appearance, since training text may look different from the rendered glyphs.Source: §3.2 · p. 3

3.3 DEPTH-NORMALIZED RECOGNIZER FEATURE SUPERVISION

• Depth-normalized supervision compares decoded text-region features with ground-truth features at shallow, middle, and deep recognizer depths, balancing their contributions.The recognizer and VAE decoder are frozen, and the supervision gradient propagates through the decoder to the video model.Source: §3.3 · pp. 4–5

3.4 TRAINING AND INFERENCE METHOD

• Training freezes the base video model and text encoders while tuning rank-32 LoRA and the glyph-embedding projector; inference edits clips as a whole or processes multiple trajectories sequentially.For recognizer supervision, only the first 9 predicted frames are decoded; training uses 1,050 real-scene clips with 1,430 trajectories disjoint from VTEdit.Source: §3.4 · pp. 5–6

4 VTEDIT: VIDEO TEXT EDITING BENCHMARK

• VTEdit contains 288 real-scene clips with 440 annotated text trajectories, disjoint from training and trimmed to at most 49 frames.The clips come from 4K public social-media videos and are cropped and resized to 720 × 1280.Source: §4 · p. 6
• Text replacement covers 398 trajectories, replacing existing text with a target of similar length.The benchmark also includes text addition on 42 trajectories in 30 clips, with boxes manually drawn in the first frame and propagated by tracking.Source: §4 · p. 6
• The 42 text-addition trajectories write new text onto text-free surfaces, complementing the replacement task.Targets for both tasks are generated by GPT-5.Source: §4 · p. 6
• Annotations track text in quadrilateral boxes across frames; text is detected with PP-OCRv6 and associated across frames.Each clip is captioned with Qwen2.5-VL; the benchmark includes 358 Chinese and 82 English trajectories, with Chinese spanning 740 distinct characters.Source: §4 · p. 6

5.1 EXPERIMENTAL DETAILS

• Evaluation covers text accuracy, temporal consistency, background preservation, and user preference, comparing image, mask-based video, instruction-guided, and commercial editing methods.Text metrics use PP-OCRv6; background metrics are measured outside text boxes, and six participants judge 100 VTEdit cases.Source: §5.1 · p. 7

5.2 COMPARISON WITH STATE-OF-THE-ART

• Sen.ACC improves from 0.8393 to 0.9408 and NED from 0.9307 to 0.9862 versus FLUX-Text; our method also leads in background preservation and user preference.Its warping error is 1.5433, close to Seedance’s 1.4862; it achieves PSNRbg of 36.11 dB, SSIMbg of 0.9736, and 64.3% of user-study votes.Source: §5.2 · p. 7, Table 1 · p. 8
• Across text addition and replacement examples, our method renders correct text aligned with the editing region, while competing methods show garbling, mismatches, or temporal variation.The comparisons include commercial outputs that may be legible but fail to match the target or scene.Source: §5.2 · p. 8, Figure 6 · p. 9

5.3 ABLATION STUDIES

• Removing the glyph reference lowers Sen.ACC from 0.9408 to 0.9013 and NED from 0.9862 to 0.9658; across ablations, this reference brings the largest text-accuracy gain.The no-reference variant erases the editing region and supplies the target text only as a text condition.Source: §5.3 · p. 8, Table 2 · p. 8
• Multi-depth recognizer supervision raises Sen.ACC from 0.9282 for the best single depth to 0.9408; multi-depth VGG features improve accuracy less.The study compares no supervision, single-depth recognizer features, and multi-depth features.Source: §5.3 · p. 8, Table 3 · p. 9
• Without depth-wise normalization, Sen.ACC is 0.9270; with normalization, it reaches 0.9408, as unnormalized depth combination brings no clear gain over a single depth.The unnormalized result is within the single-depth range of 0.9241 to 0.9282.Source: §5.3 · p. 9, Table 4 · p. 9

6 CONCLUSION

• On VTEdit, the method achieves the best text accuracy and background preservation among image text editing methods, video editing methods, and commercial models, and receives the highest user preference.Source: §6 · p. 10
• The method provides per-frame glyphs that follow the text’s position and perspective through a trajectory-aligned rendering reference.Source: §6 · p. 10
• Depth-normalized recognizer feature supervision uses multi-depth features from a frozen text recognizer to supervise generated text.Source: §6 · p. 10
• The method casts video text editing as text-conditioned video inpainting along a text trajectory.Source: §6 · p. 10

A ADDITIONAL QUALITATIVE COMPARISONS

• The method renders the correct target text, keeps it consistent across frames, and preserves content outside the editing region.Source: A ADDITIONAL QUALITATIVE COMPARISONS · p. 13
• The qualitative comparisons cover text accuracy, temporal consistency, and background preservation.Source: A ADDITIONAL QUALITATIVE COMPARISONS · p. 13
• Each compared method falls short in at least one of text accuracy, temporal consistency, or background preservation.Source: A ADDITIONAL QUALITATIVE COMPARISONS · p. 13

A.1 COMPARISON ON TEXT ACCURACY

• The method accurately renders the target text with clear, well-formed glyphs.Source: §A.1 · p. 13, Figure 7 · p. 13
• Seedance produces legible text that does not match the target, while FLUX-Text produces readable text with distorted glyphs.Source: §A.1 · p. 13
• VACE, Kling O3, and Kiwi-Edit produce incorrect characters; UniVideo removes the original text, and VideoPainter substitutes a map graphic.Source: §A.1 · p. 13
• Figure 7 marks editing regions with green boxes and unsuccessful edits and character errors with red boxes.Source: Figure 7 · p. 13

A.2 COMPARISON ON TEMPORAL CONSISTENCY

• The method accurately renders the target text while maintaining a consistent lettering style across frames.Source: §A.2 · p. 13
• FLUX-Text produces clear, accurate text in individual frames but shows inconsistent lettering style and pronounced flickering across frames.Source: §A.2 · p. 13
• Seedance renders the target text correctly in some frames but has glyph errors in others.Source: §A.2 · p. 13
• Other video editing methods fail to replace the original text or produce incorrect characters or noticeable blurring.Source: §A.2 · p. 13

A.3 COMPARISON ON BACKGROUND PRESERVATION

• Figure 8 compares frames 1, 6, 11, and 21; our method maintains legible lettering and a consistent layout across them.Green boxes mark source editing regions, and red boxes highlight failed edits and inconsistent text appearance.Source: Figure 8 · p. 14
• Seedance, Kiwi-Edit, Kling O3, VACE, VideoPainter, FLUX-Text, and UniVideo introduce unintended background changes, including altered packaging and malformed or blurred text.Seedance changes a neighboring package’s product name, illustration, and color scheme; Kiwi-Edit distorts its illustration, and Kling O3 modifies neighboring product text.Source: §A.3 · pp. 13–14
• Figure 9 shows our method more faithfully preserving content outside the editing region than competing methods.The comparison highlights surrounding text, illustrations, and other visual details.Source: §A.3 · pp. 13, 15, Figure 9 · p. 15

B IMPLEMENTATION DETAILS OF COMPARED METHODS

• FLUX-Text and mask-based methods use text boxes as the editing region and the target text in the prompt.Source: B IMPLEMENTATION DETAILS OF COMPARED… · p. 15
• Instruction-guided methods receive the full source video and an edit instruction that specifies the text replacement or addition location.For additions, Kling O3 and Seedance also receive a reference frame with the editing region marked.Source: B IMPLEMENTATION DETAILS OF COMPARED… · p. 15
• Each method runs at its default resolution and frame count, and its output is aligned to source-video frames for evaluation.Source: B IMPLEMENTATION DETAILS OF COMPARED… · p. 15