MedGen: Unlocking Medical Video Generation by Scaling Granularly-annotated Medical Videos

1 Introduction

• Medical videos require precise anatomical structures, accurate surgical steps, and realistic physiological dynamics, raising demands on fidelity, semantic correctness, and temporal coherence.Source: §1 · p. 1
• Existing models often produce anatomical inconsistencies, tool misuse, and implausible clinical scenarios, while available medical datasets are limited in scale, scope, and descriptive captions.These datasets are poorly suited to training or evaluating text-to-video models that rely on rich semantic input.Source: §1 · p. 1
• MedVideoCap-55K contains over 55K curated clips across real-world medical scenarios, each paired with a descriptive caption.The dataset is described as the first large-scale medical video dataset specifically designed for text-to-video generation.Source: §1 · p. 2, Table 1 · p. 2
• MedGen significantly outperforms other open-source models in visual quality and medical accuracy, while the paper introduces medical-specific evaluation metrics.The evaluation protocols include automatic metrics and expert-based assessments.Source: §1 · p. 2

2 Motivations

• Commercial generators frequently produce medical hallucinations, including incorrect anatomy, instrument misuse, and implausible clinical scenes.The paper describes these failures as reflecting a mismatch between general visual patterns and medical semantics and constraints.Source: §2 · p. 2
• Existing medical datasets often focus on narrow tasks and rarely include descriptive captions, limiting their suitability for text-to-video training and evaluation.Text-to-video models require semantic understanding and rich supervision.Source: §2 · p. 2
• MedVideoCap-55K contains more than 55K curated medical clips across diverse domains, paired with textual descriptions for natural-language-driven generation tasks.The source collection comprises over 25 million public videos.Source: §2 · p. 2

3.1 Data Construction

• From 25 million YouTube videos, keyword and metadata-classifier filtering identified 37K medical videos; channel expansion produced 140K videos totaling 10,269 hours.Source: §3.1 · pp. 2–3
• A frame classifier and adjacent-frame CLIP similarity were used to segment videos; retaining clips above 480p yielded 111K medical clips.Videos were sampled at 1 frame per second for frame classification.Source: §3.1 · p. 3
• GPT-4o generated detailed captions from eight sampled frames, video metadata, and transcripts, then generated brief captions for input-length-constrained models.Each dataset item pairs a medical clip with brief and detailed captions.Source: §3.1 · p. 3, Figure 2 · p. 3

3.2 Data Filtering and Quality Refinement

• The quality pipeline targets black borders, persistent subtitles, low aesthetic scores, technical artifacts, and clips failing a combined aesthetic-and-technical threshold.It uses Canny edges and Hough lines for borders, OCR for subtitles, LAION aesthetic scores, and Dover technical scores.Source: §3.2 · pp. 3–5

3.3 Data Statistics

• Most clips are 6–10 seconds long, have brief and detailed captions, and show medium-to-high aesthetic and technical quality scores.The paper describes this duration as balancing semantic richness with training efficiency and fitting current video-generation model limits.Source: Figure 5 · p. 4, §3.3 · p. 5
• MedVideoCap-55K spans clinical procedures, medical education, scientific communication, animations, and imaging.The paper says this breadth supports training models to generalize across real-world medical contexts.Source: Figure 5 · p. 4, §3.3 · p. 5

3.2 Data Filtering and Quality Refinement

• Each filtering stage was checked on 200 randomly sampled videos and deemed successful when at least 95% were clean or only slightly noisy.Figure 4 visualizes the filtering stages and volumes removed and retained.Source: §3.2 · p. 4, Figure 4 · p. 4

4.1 Medical Video Generation Model: MedGen

• MedGen is built on HunyuanVideo and trained on MedVideoCap-55K; evaluation compares 15 open-source and 4 commercial models using Med-VBench and three doctors’ ratings.Doctors rated text alignment, medical accuracy, and visual quality.Source: §4.1 · pp. 5–6
• Table 3 compares MedGen and HunyuanVideo on data augmentation performance in downstream supervised tasks.Source: Table 3 · p. 6

4.2 Main Results

• MedGen achieves the best Med-VBench performance among open-source models and excels in factual consistency, text-to-video alignment, and imaging quality, while scoring among the lowest in warping error.Doctor evaluations also report strong performance in text alignment, medical accuracy, and visual quality, with inter-rater Cohen’s Kappa above 0.75.Source: §4.2 · p. 6, Figure 6 · p. 6

4.3 Transferability of MedVideoCap-55K on Mochi

• Further training Mochi-1-preview on MedVideoCap-55K improves warping error, image quality, and subject consistency, but slightly reduces background consistency and motion smoothness.The authors hypothesize that adding a balanced proportion of non-medical video data could mitigate the declines; they leave this for future work.Source: Table 4 · p. 6, §4.3 · p. 7

5.1 Application I: Data Augmentation

• MedGen augmentation raises F1 by up to +15.3 on HyperKvasir and +11.7 on SurgVisDom, outperforming HunyuanVideo across three classification benchmarks.The comparison includes MedVidCL, HyperKvasir, and SurgVisDom, and reports consistently higher gains across all metrics and datasets.Source: §5.1 · p. 7

5.2 Application II: Science Popularization and User Simulation

• MedGen generates videos for science popularization, surgical simulation, medical imaging, education, and patient simulation.The paper identifies dataset augmentation, simulation prototyping, and medical communication as potential uses, particularly where real video is scarce, privacy-sensitive, or costly to obtain.Source: §5.2 · p. 7
• The generated videos are described as visually coherent and medically relevant, supporting their use in dataset augmentation and simulation prototyping.The paper also identifies potential use in medical communication.Source: §5.2 · p. 7

Text-to-Video Generation

• General text-to-video models remain limited in medical applications because medical video data are scarce, constraining the professionalism and practicality of generated content.The passage surveys commercial models including Kling and Hailuo, and open-source systems including LTX-Video, CogVideoX, and Mochi-1.Source: §6 · p. 7

Generation Models in Medical Domain

• Prior medical generators include Endora for endoscopy simulation, MedSora for medical video generation and downstream classification, and Surgen and SurgSora for surgical videos.MedSora combines spatio-temporal Mamba modules, optical flow alignment, and frequency-compensated video VAEs.Source: §6 · p. 7
• Unlike prior medical generation work, this paper targets models serving a wide range of medical and healthcare needs.Source: §6 · p. 8

Text-to-Video Datasets

• General text-video datasets are not directly applicable to medical use, while existing medical video datasets are often used for supervised classification.Examples of general datasets include OpenVid, Panda, and WebVid; medical datasets include MedVidCL, Colonoscopy, Kvasir-Capsule, and CholecTriplet.Source: §6 · p. 8

7 Conclusion

• MedVideoCap-55K provides over 55,000 curated clips across real-world medical scenarios as a foundation for generalist medical video generation models.The conclusion describes the dataset as large-scale, diverse, and caption-rich.Source: §7 · p. 8
• MedGen achieves state-of-the-art performance among open-source models and rivals commercial systems across multiple benchmarks.The conclusion reports utility for downstream medical video analysis through data augmentation and significant performance gains.Source: §7 · p. 8
• The paper identifies domain-specific datasets and tailored evaluation as important to advancing medical video generation.Source: §7 · p. 8
• The dataset is intended strictly for research, not clinical practice, and its publicly available videos exclude personal or sensitive content.Captions were generated with MLLMs and manually verified to remove inappropriate or identifiable material.Source: §7 · p. 8

A.1 Training Details on MedGen

• Table 5 provides the detailed training configuration used to train MedGen.Source: §A.1 · p. 11, Table 5 · p. 11
• Figure 8 compares training-loss changes and training time across training strategies.Source: Figure 8 · p. 11
• Under the same number of training steps, LoRA requires only half the GPU runtime of full-parameter fine-tuning.Figure 8 compares loss changes and training time for the strategies.Source: §A.1 · p. 11, Figure 8 · p. 11

• Before February 10, 2025, tests used Pika 2.0, Kling 1.6, and Hailuo video-01.Source: §A.2 · p. 11

A.3 Evaluation Details

• For 200 medical prompts, models were run once with default settings, and three evaluators’ distortion ratings were aggregated into warping error.Source: §A.3 · p. 11
• Three evaluators independently rated each video’s distortion, with at least two ratings determining whether it was acceptable.No Distortion or Minor Distortion ratings counted as acceptable; two Moderate or Severe ratings meant unacceptable.Source: §A.3 · p. 11, Table 7 · p. 12, Figure 9 · p. 13
• Evaluator-pair Cohen’s Kappa scores of 0.82, 0.79, and 0.77 indicated strong consistency in distortion ratings.Source: §A.3 · p. 11
• Three doctors anonymously compared MedGen with a randomly selected model on matched prompts, judging text alignment, medical accuracy, and visual quality.Doctors could choose “Both Loss” when neither video performed satisfactorily.Source: §A.3 · p. 12

A.4 VideoScore Evaluation Result

• VideoScore supplementary evaluations further validate MedGen’s competitiveness in mainstream video generation tasks.Source: §A.4 · pp. 12–13, Table 8 · p. 13

A.5 Inter-Rater Reliability Among Expert Evaluators

• 0.83, 0.81, and 0.78 were the Cohen’s Kappa values for expert pairs 1–2, 1–3, and 2–3, respectively.The authors interpret these values as strong agreement and evidence of scoring reliability.Source: §A.5 · pp. 13–14, Table 9 · p. 14

A.6 The generalizability of MedGen

• A HunyuanVideo model trained on 10k medical teaching videos showed strong generalization on unseen medical video categories.Source: §A.6 · p. 14, Table 10 · p. 14