How Far Are Surgeons from Surgical World Models? A Pilot Study on Zero-shot Surgical Video Generation with Expert Assessment

1 Introduction

• Surgery tests specialized understanding because instrument movements, tissue responses, and procedural actions involve complex causal interactions under precise spatiotemporal constraints.The paper contrasts this expertise with general physical understanding learned from natural videos.Source: §1 · p. 2
• SurgVeo evaluates zero-shot 8-second continuations from laparoscopic and neurosurgical clips using four board-certified surgeons and the four-tier SPP.The tiers progress from visual appearance through instrument action and environmental feedback to surgical strategy.Source: §1 · p. 2, Figure 1 · p. 3
• Veo-3 produces visually convincing surgical videos but struggles with instrument operation, tissue feedback, and inferring surgical intent.The results identify a gap between visual mimicry and the causal knowledge needed for expert-level understanding.Source: §1 · p. 2
• The authors identify surgical training, preoperative planning, intraoperative guidance, and autonomous robotics as potential clinical applications, and plan to release SurgVeo and expert feedback.Source: §1 · p. 3

2 The SurgVeo Benchmark

• The benchmark pairs each surgical input frame with its following reference video, and the study evaluates Veo-3 continuations with expert surgeons.The workflow includes data preparation, zero-shot generation, and multidimensional expert evaluation.Source: §2 · p. 3, Figure 2 · p. 4

2.1 Surgical Data Preparation

• SurgVeo is tailored to test whether video-generation models capture the specialized dynamics, causal relationships, and knowledge of surgery.The benchmark is distinguished from natural-scene and general video-quality benchmarks.Source: §2.1 · p. 3

2.1.1 Rationale and Selection of Surgical Procedures

• The two tracks use laparoscopic hysterectomy and endoscopic pituitary surgery to represent distinct procedural challenges and causal relationships.The procedures test confined-space instrument interaction and soft-tissue deformation, as well as delicate maneuvers near critical neurovascular structures.Source: §2.1.1 · pp. 3–4

2.1.2 Data Sourcing and Structure

• The benchmark contains 50 clips from three hysterectomy recordings and three pituitary-surgery recordings, covering multiple stages of both procedures.Each sample pairs an input frame preceding an action with the immediately following reference video; the benchmark is slated for public release.Source: §2.1.2 · pp. 4–5

2.2.1 Zero-Shot Video Generation Task

• Veo-3, without surgical-video fine-tuning, generates an 8-second continuation from a single input frame and text prompt.The limited input is intended to test whether the model can extrapolate surgical dynamics and procedural logic.Source: §2.2.1 · p. 5

2.2.2 Prompting Strategy

• The baseline prompt supplies procedure type and general visual requirements; the stage-aware prompt additionally names the current surgical stage.The paired conditions test whether explicit stage information improves plausibility or the model can infer next steps from visual context.Source: §2.2.2 · p. 5

2.3 Expert Evaluation

• Surgeons assess generated videos against real surgical references to evaluate reasoning, instrument operation, and tissue biomechanics beyond superficial visual quality.Source: Figure 2 · p. 4, §2.3 · p. 5

2.3.1 Evaluation Panel

• Two laparoscopic surgeons assess the hysterectomy track, while two neurosurgeons assess the pituitary-surgery track independently.The authors argue that expert judgment captures subtle aspects of plausibility that automated metrics cannot.Source: §2.3.1 · p. 5

3.1 Performance Across the Surgical Plausibility Pyramid

• At 8 seconds, stage-aware laparoscopy scored 1.69 ± 0.20 for Instrument Operation, 1.53 ± 0.12 for Environment Feedback, and 1.81 ± 0.20 for Surgical Intent, contrasting with high visual plausibility.The reported high visual scores and low higher-level scores establish the study’s central plausibility gap.Source: §3.1 · p. 7, Figure 3 · p. 8, Figure 4 · p. 8

3.2 Temporal Degradation of Plausibility

• 3.06 ± 0.08 at 1 second fell to 1.64 ± 0.12 at 8 seconds for baseline laparoscopic Environment Feedback; Surgical Intent fell from 3.11 ± 0.16 to 1.61 ± 0.16.Visual plausibility stayed relatively high and stable, while the three higher-level dimensions declined as the prediction horizon extended.Source: §3.2 · pp. 7–8

3.3 Discrepancies Between Surgical Specialties

• At 1 second under baseline prompting, Instrument Operation scored 3.36 ± 0.20 in laparoscopy versus 2.77 ± 0.02 in neurosurgery; higher-level scores were typically lower in neurosurgery.The authors suggest neurosurgery’s precision, delicate tissue handling, and microscopic scale make its implicit rules more complex for Veo-3 to capture.Source: §3.3 · p. 8, Figure 3 · p. 8, Figure 4 · p. 8

3.4 Ineffectiveness of Stage-Aware Prompting

• Stage-aware context produced no significant or consistent plausibility gains; at 1 second, laparoscopic Instrument Operation was 3.22 ± 0.16 versus 3.36 ± 0.20 with baseline prompting.The prompt comparison’s violin plots show similar distributions across plausibility dimensions for both specialties.Source: §3.4 · p. 8, Figure 3 · p. 8, Figure 4 · p. 8

3.5 Video Error Analysis: Qualitative Examples and Quantitative Distribution

• Qualitative examples span fabricated instruments, procedurally wrong movements or targets, implausible tissue feedback, and misread surgical intent.Figure 5 pairs real and generated frames and marks specific illogical, anatomically incorrect, or physically impossible artifacts.Source: §3.5 · p. 9, Figure 5 · p. 9
• Visual distortions made up 6.2% of laparoscopic and 2.8% of neurosurgical failures; four dominant high-level error types alone constituted 83% of neurosurgical errors.Across both tracks, the four dominant categories were surgical intent, environment feedback, surgical instrument, and inappropriate operation errors.Source: §3.5 · p. 10, Figure 6 · p. 10

• Veo-3 renders surgical scenes with striking visual fidelity but does not understand the causal principles governing them.Source: §4 · p. 10
• Explicit surgical-stage context did not significantly improve plausibility, suggesting the deficit is not simply missing information.The discussion argues that the model cannot translate concepts such as vessel ligation into physically sound actions and consequences.Source: §4 · p. 10
• Surgical world models require expert knowledge of anatomy, physiology, and biomechanics beyond everyday physical understanding.Source: §4 · p. 10
• The authors argue that scaling general-domain training alone will be insufficient; future models may need structured domain knowledge and hard physical and logical constraints.Source: §4 · p. 11
• A true surgical world model could support interactive training, pre-operative rehearsal, and intra-operative guidance; SurgVeo quantifies the current performance gap as a first step toward that future.Source: §4 · p. 11
• The proposed research directions pair explicit surgical knowledge with improved modeling of physical laws and long-range causal dependencies.The authors also identify physics-informed modeling and methods to mitigate error accumulation over time as priorities.Source: §4 · p. 11

• Current models reproduce surgery photorealistically but fail to follow basic principles of surgical action, consequence, and strategy.Source: §5 · p. 11
• SurgVeo and the Surgical Plausibility Pyramid provide a standardized, clinically grounded framework to measure and guide progress in surgical world modeling.Source: §5 · p. 11
• The authors frame this work as a foundation and roadmap for moving beyond visual mimicry toward models with deeper causal understanding.Source: §5 · p. 11

• SurgVeo has separate laparoscopic and neurosurgery tracks, covering distinct surgical environments and procedural complexities.Source: A Tracks of SurgVeo Benchmark · p. 14
• The laparoscopic track features endoscopic abdominal views and instruments such as graspers, scissors, and electrocautery, with limited visibility and indirect tissue manipulation.Source: A Tracks of SurgVeo Benchmark · p. 14
• The neurosurgery track uses high-magnification microscopic views and specialized instruments for delicate work on neural and vascular structures.The passage notes the need for exceptional precision around critical anatomy, where millimeter-scale errors can have significant clinical consequences.Source: A Tracks of SurgVeo Benchmark · p. 14

• The benchmark contains 50 video segments: 18 in the laparoscopic track and 32 in the neurosurgery track.Source: B Statistics of SurgVeo Benchmark · p. 14
• The laparoscopic track covers seven major stages, while neurosurgery covers 12, including tumor excision and graft placement.Source: B Statistics of SurgVeo Benchmark · p. 14
• Not every surgery includes every stage, reflecting variability in surgical workflows.Source: B Statistics of SurgVeo Benchmark · p. 14

C Data Structure of SurgVeo Benchmark

• Each SurgVeo sample pairs a pre-action input frame with an 8-second generated continuation and an 8-second real reference for expert comparison.The benchmark shows stage distributions for both tracks, while noting that not every surgery includes every possible stage.Source: C Data Structure of SurgVeo Benchmark · pp. 14–15, Figure 7 · p. 14

D Task Formulation of Surgical Video Generation

• The task tests whether Veo-3 can predict an 8-second surgical continuation from minimal visual context, with either baseline or stage-aware prompting.Baseline prompts omit the current stage; stage-aware prompts name it to test whether explicit context improves stage-appropriate actions.Source: D Task Formulation of Surgical Video… · p. 15

E.1 Laparoscopic Surgery Track

• Laparoscopic prompts request a realistic hysterectomy continuation with clear anatomy, cool lighting, smooth camera adjustments, and responsive tissues.The stage-aware variant adds the current surgical stage to the baseline prompt.Source: §E.1 · pp. 15–16

E.2 Neurosurgery Track

• Neurosurgery prompts request a realistic endoscopic pituitary-surgery continuation, specifying clear anatomy, cool lighting, smooth camera adjustments, and responsive tissues.The stage-aware variant adds the current surgical stage to the baseline prompt.Source: §E.2 · p. 16

F Implementation Details of Surgical Video Generation

• Veo-3.0-quality generations use each SurgVeo input frame with either a baseline or stage-aware prompt, without fine-tuning or domain-specific adaptation.The model is accessed through Google Flow and generates 8-second continuations at default resolution and frame rate.Source: F Implementation Details of Surgical… · p. 16

G.1 Visual Perceptual Plausibility

• Visual Perceptual Plausibility rates clarity, lighting, tissue appearance, motion smoothness, and artifacts or discontinuities.The top rating describes clear, stable video with smooth motion that is visually indistinguishable from high-quality surgical recordings.Source: §G.1 · p. 16

G.2 Instrument Operation Plausibility

• Instrument Operation Plausibility assesses instrument realism and whether selection, trajectories, manipulation, force, and coordination are technically sound.The lowest rating includes fabricated or physically impossible instruments and tools used for actions they are not designed to perform.Source: §G.2 · p. 17

G.3 Environment Feedback Plausibility

• Environment Feedback Plausibility judges whether tissue and organ responses to actions are physically and anatomically realistic.The rubric considers consequences such as deformation, bleeding, and eschar formation against the reference video.Source: §G.3 · p. 17

G.4 Surgical Intent Plausibility

• Surgical Intent Plausibility evaluates whether predicted actions show a coherent purpose and strategic progression for the procedure.The rubric compares actions with appropriate procedural steps and clinical decision-making against the reference video.Source: §G.4 · p. 18

H Qualitative Case Studies of Generated Videos

• The case studies compare representative high- and low-scoring generations with synthesized expert feedback to add detail beyond aggregated scores.They present the examples as an examination of model performance across the benchmark.Source: H Qualitative Case Studies of… · p. 18

H.1 Analysis of High-Scoring Examples

• High-scoring examples show Veo-3 most capable of mimicking visually plausible simple or non-operative actions, including dissection and a pause.Experts described the dissection as relatively natural with few errors, and the pause as almost identical to its reference.Source: §H.1 · p. 19

H.2 Analysis of Low-Scoring Examples

• Low-scoring examples demonstrate catastrophic failures in surgical logic that can occur despite acceptable visual quality.The examples illustrate the quantitative higher-level shortcomings through specific cases.Source: §H.2 · p. 19
• In a laparoscopic suturing attempt, Veo-3 misses the intended action, hallucinates a non-existent instrument, and produces poor tissue feedback.Experts describe the operation as unrecognizable and non-standard, with errors cascading from misunderstanding the action and tools.Source: §H.2 · p. 19
• In neurosurgical glue application, Veo-3 renders acceptable visuals but misses the intent and generates unrelated actions, instruments, and feedback.The case illustrates failure to translate the strategic purpose into a coherent sequence of actions and consequences.Source: §H.2 · p. 19

• The release includes input frames, real surgical continuations, Veo-3 outputs, anonymized scores across four dimensions, and evaluation protocols.Source: I Public Release and Reproducibility · p. 19
• The release is intended to support reproducing the findings, benchmarking alternative generative models, and studying world modeling in specialized professional domains.Source: I Public Release and Reproducibility · p. 19
• Examples range from plausible dissection and near-identical neurosurgical video to a fabricated instrument and a missed biologic-glue intent.Source: Figure 8 · p. 20, Figure 9 · p. 21