Source-linked AI summary
Code2Video: A Code-centric Paradigm for Educational Video Generation
Yanzhe Chen, Kevin Qinghong Lin, Mike Zheng Shou
TL;DR
Educational video generation must convey domain knowledge with coherent transitions and precise visual structure, challenging pixel-space methods. Code2Video uses Planner, Coder, and Critic agents over executable code and introduces MMMC with multidimensional evaluation. It reports a 40% improvement over direct code generation and videos comparable to human-crafted tutorials.
Problem
Educational video generation requires domain expertise, coherent transitions, and precise spatial structure beyond what existing pixel-space methods reliably provide.
Method
Code2Video uses Planner, Coder, and Critic agents to create educational videos through executable code, while MMMC evaluates aesthetics, efficiency, and knowledge transfer.
Results
40% improvement over direct code generation is reported, with generated videos comparable to human-crafted tutorials.
Takeaways & Limitations
The paper establishes executable code as a scalable, interpretable, and controllable medium for structured educational video generation.
Abstract
from arXiv · showhide
While recent generative models advance pixel-space video synthesis, they remain limited in producing professional educational videos, which demand disciplinary knowledge, precise visual structures, and coherent transitions, limiting their applicability in educational scenarios. Intuitively, such requirements are better addressed through the manipulation of a renderable environment, which can be explicitly controlled via logical commands (e.g., code). In this work, we propose Code2Video, a code-centric agent framework for generating educational videos via executable Python code. The framework comprises three collaborative agents: (i) Planner, which structures lecture content into temporally coherent flows and prepares corresponding visual assets; (ii) Coder, which converts structured instructions into executable Python codes while incorporating scope-guided auto-fix to enhance efficiency; and (iii) Critic, which leverages vision-language models (VLM) with visual anchor prompts to refine spatial layout and ensure clarity. To support systematic evaluation, we build MMMC, a benchmark of professionally produced, discipline-specific educational videos. We evaluate MMMC across diverse dimensions, including VLM-as-a-Judge aesthetic scores, code efficiency, and particularly, TeachQuiz, a novel end-to-end metric that quantifies how well a VLM, after unlearning, can recover knowledge by watching the generated videos. Our results demonstrate the potential of Code2Video as a scalable, interpretable, and controllable approach, achieving 40% improvement over direct code generation and producing videos comparable to human-crafted tutorials. The code and datasets are available at https://github.com/showlab/Code2Video.
1 INTRODUCTION
Educational video generation requires domain knowledge, coherent temporal sequencing, and spatially clear animations, challenges that pixel-space approaches struggle to address. Code2Video responds with a transparent tri-agent framework and MMMC’s multidimensional evaluation protocol.
- Educational videos must combine domain expertise, logically ordered reasoning, designed transitions, and legible spatial organization.
- Code-centric pipelines offer scalable, interpretable, and controllable generation through modular assets and explicitly scripted sequencing and layout.
- Code2Video assigns planning, executable-code generation, and multimodal spatial refinement to the Planner, Coder, and Critic.
- MMMC evaluates educational videos through aesthetics and structure, code efficiency, and TeachQuiz knowledge transfer.
- The framework’s contributions include a code-centric paradigm, modular visual-animation agents, and a multidimensional benchmark for educational video generation.
2 RELATED WORK
Prior work provides strong pixel-space synthesis and increasingly modular LLM agents, but educational video generation requires finer control, executable grounding, and evaluation of knowledge transfer. Code2Video addresses these needs through Manim-based generation and metrics spanning appearance, teaching effectiveness, and efficiency.
- Pixel-space video models achieve perceptual fidelity but provide limited control over precise layout and symbolic alignment needed for educational content.
- LLM tool-use research supports modular workflows through API calls, code execution, retrieval, verification, and project-level generation.
- Code-centric educational generation maps learning queries to executable Manim code, emphasizing multistep reasoning, temporal sequencing, and spatial coherence.
- MMMC contains 117 long-form videos and 339 timestamped segments across 13 subject areas, balancing long-horizon reasoning with fine-grained supervision.
- The evaluation combines VLM-based aesthetic judgments, TeachQuiz knowledge transfer, and code-generation efficiency.
4 METHOD: CODE2VIDEO
Code2Video converts a learning query into a structured, executable, and visually refined Manim video through coordinated planning, coding, debugging, and critique. Its design uses storyboards, cached assets, scoped repair, and visual-anchor feedback to preserve temporal and spatial coherence.
- 4.1 PLANNER: QUERY TO STORYBOARD: The Planner converts a topic into an outline and storyboard that organize concepts, lecture lines, examples, and paired animations.
- 4.1 PLANNER: QUERY TO STORYBOARD: The Planner retrieves reference images and reusable visual assets, caching them to reduce hallucination, avoid redundant generation, and maintain consistency.
- 4.2 CODER: STORYBOARD TO CODE: The Coder translates storyboard sections and shared assets into executable Manim code, with parallel section-level synthesis reducing the cost of full-video generation.
- 4.2 CODER: STORYBOARD TO CODE: ScopeRefine repairs errors hierarchically at line, block, and global section scopes instead of repeatedly processing complete code and logs.
- 4.3 CRITIC: EFFECTIVE VISUAL REFINEMENT: The Critic combines rendered-video inspection, occupancy tracking, visual-anchor prompts, and VideoLLM feedback to refine spatial placement and clarity.
5 EXPERIMENT
Code2Video is evaluated against human-crafted, pixel-based, direct code, and agentic baselines using efficiency, aesthetic quality, knowledge transfer, qualitative comparisons, ablations, and human judgments. The results show stronger controllability and teaching quality than pixel-based or direct code generation, while revealing remaining limitations in some aesthetic dimensions and human sensitivity to layout and duration.
- 5.2 MAIN RESULTS: 50% AES and 46% TeachQuiz improvements are achieved with Claude Opus 4.1 over the corresponding direct code-centric baseline.Code2Video shows stable gains across backbone LLMs, although human-crafted videos remain strong.
- 5.2 MAIN RESULTS: Pixel-based models obtain the lowest AES and TeachQuiz scores, particularly on LF, because text grounding, timing, and cross-frame coherence are weak.Direct code-centric generation improves over pixel-based models, supporting code as a controllable medium for educational videos.
- 5.2 MAIN RESULTS: Code-driven synthesis produces clearer text and formulas, stable non-occluding layouts, and stepwise lecture alignment compared with Veo3.Veo3 often exhibits blurry or corrupted text, inconsistent styles, and drifting visuals that weaken semantic grounding.
- 5.3 ABLATION STUDIES: Removing the Planner collapses TeachQuiz and Aesthetics by approximately 41 points, while the External Database, Visual Anchors, and Critic provide complementary gains.TeachQuiz is more sensitive than Aesthetics, exposing knowledge-transfer gaps even when videos remain visually acceptable.
- 5.3 ABLATION STUDIES: 15.4 → 86.6 minutes of latency follows removal of parallel execution, while scope-aware repair avoids the higher correction cost of retry and full-code debugging.Parallel synthesis and localized ScopeRefine repair are presented as essential for scalable generation.
- 5.3 ABLATION STUDIES: Human ratings show sharper quality contrasts, greater sensitivity to brief occlusions, and reduced engagement when videos require sustained attention over longer durations.These patterns broadly align with VLM-based trends but expose frame-level layout and duration effects that VLMs may miss.
6 CONCLUSION
The paper establishes executable code as a unifying medium for temporal sequencing and spatial organization in educational video generation. Code2Video and MMMC provide a controllable, interpretable framework and evaluation path, with future work targeting broader video scope and lighter, more scalable agents.
- 6 CONCLUSION: Code2Video establishes executable code as the unifying medium for temporal sequencing and spatial organization in educational video generation.The framework supports controllable and interpretable generation with multimodal feedback.
- 6 CONCLUSION: MMMC evaluates code-driven educational videos across efficiency, aesthetics, and knowledge transfer, charting a path for structured and interpretable content generation.The paper identifies broader video coverage and lightweight, scalable agents as future directions.
A.1.1 UNLEARNING DETAILS AND TEACHQUIZ
TeachQuiz evaluates whether videos enable knowledge recovery after a VLM is prompted to suppress prior knowledge. Its ablations show that aligned text and animation jointly produce the strongest gains.
- Evaluation Setup: The evaluation uses 10 four-option multiple-choice questions per topic, including visually grounded reasoning items.Questions test alignment between conceptual knowledge and its visual instantiation, rather than definitions alone.
- TeachQuiz Protocol: TeachQuiz measures video-induced knowledge recovery by comparing relearning accuracy after unlearning against the unlearned baseline.S1 measures accuracy with prior knowledge blocked, while S2 measures accuracy after viewing video V; higher TQ indicates stronger acquisition.
- Evidence-Source Ablation: Text-only supervision improves performance moderately but remains below full video-based relearning, showing that textual scaffolding alone is insufficient.
- Evidence-Source Ablation: Animation-only inputs also improve accuracy over unlearning but underperform the full condition, indicating that visual cues benefit from textual grounding.
- Evidence-Source Ablation: Random-topic videos return performance to the unlearn level or lower, while combined text and animation yield the strongest TeachQuiz gains.This supports the role of semantically aligned educational content rather than superficial video exposure.
A.1.2 HUMAN STUDY: MIDDLE SCHOOL VS. UNDERGRADUATE COMPARISON
The human study compares middle school and undergraduate learners on aesthetics, TeachQuiz, and completion willingness. Code2Video benefits learners with less prior knowledge while maintaining engagement across both groups.
- Learner Comparison: Code2Video raises middle school TeachQuiz to 88.1 versus 55.0 for undergraduates, consistent with larger gains among learners closer to an unlearned state.
- Learner Comparison: Across both learner groups, Code2Video achieves high aesthetics and completion willingness while outperforming pixel-based models by large margins.
- Learner Comparison: Shorter agentically generated videos retain strong engagement and learning outcomes, whereas long human-made videos reduce middle school completion willingness.
A.1.3 ABLATION ON VISUAL ANCHOR POINT GRANULARITY
The anchor-granularity ablation shows that structured spatial scaffolds improve layout quality, but excessive density introduces clutter. A 6 × 6 grid provides the best reported balance.
- Anchor Granularity: Moving from no anchors to 4 × 4 and 6 × 6 grids substantially improves Element Layout and Attractiveness.
- Anchor Granularity: An 8 × 8 grid degrades both metrics because overly fine discretization introduces clutter and element occlusion.
- Anchor Granularity: Self-directed placement remains far behind grid-based designs, while unconstrained heuristics produce inefficient spacing and visual imbalance.
- Anchor Granularity: A 6 × 6 anchor grid achieves the best trade-off between layout quality and aesthetics among the evaluated placement strategies.
A.1.4 EVALUATION ON THEOREMEXPLAINBENCH
Code2Video generalizes beyond lecture-line tutorials to visualization-focused theorem explanations, improving layout, relevance, and overall quality while preserving accuracy relative to TEA.
- Results: 0.32 higher Element Layout and 0.12 higher Visual Relevance are achieved with GPT-4o over TEA.Element Layout rises from 0.59 to 0.91, while Visual Relevance rises from 0.79 to 0.91.
- Results: Overall scores improve by 0.06–0.10 over TEA while Accuracy & Depth remains comparable or better.The results indicate that adding lecture lines does not reduce accuracy in the multimodal educational setting.
- Limitations: Gemini-2.0 Flash shows a backbone-specific trade-off, with Visual Consistency of 0.70 versus 0.87 despite better layout and logical flow.The comparison points to a need for backbone-specific tuning of rendering conventions and layout control.
- Method: Code2Video combines hierarchical planning, asset expansion, scope-guided coding, auto-fix, and checkpointed visual prompting.These components provide scaffolding, reliable Manim code, and discrete anchor placement to reduce clutter and misalignment.
A.1.5 DETAILS OF MMMC
MMMC is a curated benchmark of professionally produced, discipline-specific Manim tutorials, combining long-form videos, timestamped segments, and topic-level supervision. The section also documents reference-asset retrieval, quality evaluation, broad qualitative coverage, and limitations for abstract concepts.
- Data Collection: The benchmark selects topics for pedagogical relevance and executable grounding, using practitioner-created Manim references as realizability evidence.The corpus is based on the 3Blue1Brown repository, whose videos provide both teaching value and an empirical upper bound for code-driven visualization.
- Data Collection: Timestamped segments provide outline-generation supervision and short-form instructional references, while an LLM maps video metadata to concise tutorial topics.The collection process uses YouTube timestamps, titles, descriptions, and metadata to build topic mappings.
- Dataset Statistics: MMMC contains 456 tutorial videos: 117 full-length videos and 339 timestamped segments spanning 13 disciplinary categories.Full-length videos average 1014.41 seconds, while segmented clips average 201.13 seconds.
- External Database: Retrieved visual assets enhance appeal, cross-section consistency, and concept depiction, but abstract topics may lack useful visual counterparts.The Critic removes unusable assets, including black images that disappear against dark backgrounds.
- Qualitative Evaluation: Qualitative cases cover topics from Euler’s Formula to Neural Networks, while comparisons with diffusion-based models emphasize sharper, semantically aligned, pedagogically faithful outputs.Figure 8 illustrates preservation of visual clarity and pedagogical flow across diverse topics; Figure 9 compares outputs under eight-second generation limits.
- Evaluation: VLM-as-Judges evaluate videos across Element Layout, Attractiveness, Logic Flow, Visual Consistency, and Accuracy & Depth.The scoring protocol uses five dimensions to assess visual organization, engagement, temporal coherence, stylistic stability, and educational correctness.