WorldScore: A Unified Evaluation Benchmark for World Generation

1. Introduction

• World generation requires comprehensive, connected scenes with detailed spatial controls, motivating a previously absent unified benchmark spanning diverse worlds and model types.The intended capability extends beyond generating disconnected individual environments.Source: §1 · p. 1
• VBench [26] focuses on single-scene quality without explicit spatial layout control; WorldScore distinguishes failures to generate new scenes or follow instructed camera movements.Existing benchmarks also lack camera specifications and reference images needed by many 3D/4D scene-generation methods [11] [16] [39].Source: Figure 1 · p. 1, §1 · p. 1
• WorldScore decomposes world generation into sequential (current scene, next scene, layout) tasks, comparing 3D, 4D, text-to-video, and image-to-video methods through video outputs.Current scenes include both images and text prompts; layout specifications include both camera matrices and textual descriptions.Source: §1 · pp. 1–2
• 10 metrics contribute to WorldScore across controllability, quality, and dynamics.These aspects measure adherence to control inputs, fidelity and consistency, and accurate, stable motion, respectively.Source: §1 · p. 2
• 3000 test examples cover static and dynamic worlds across indoor/outdoor environments and photorealistic/stylized visual domains.Static worlds include 5 indoor and 5 outdoor categories with varying sequence lengths; dynamic worlds cover 5 dynamics types. Each example has a corresponding stylized counterpart.Source: §1 · p. 2
• 20 models undergo evaluation: 6 image-to-video, 7 text-to-video, 6 3D scene-generation, and one 4D generation model.The evaluation includes 18 open-source and 2 closed-source models and identifies insights and challenges in current world-generation approaches.Source: §1 · p. 2

Video generation benchmarks

• Unlike VBench [26] and WorldModelBench [40], WorldScore evaluates multi-scene world generation and accommodates 3D, 4D, image-to-video, and text-to-video models.These existing benchmarks lack layout control and multi-scene evaluation, and their designs are incompatible with 3D/4D methods requiring camera specifications.Source: §2 · p. 2

Video generation models

• WorldScore evaluates both text-to-video and image-to-video models through a unified design accommodating text and image conditioning.Sora has demonstrated the potential of video models for world generation.Source: §2 · pp. 2–3

3D scene generation

• Diffusion-based 3D scene methods [11] [12] sequentially outpaint 2D images and project synthesized extensions into 3D representations using depth estimation.WorldScore includes 3D and 4D generation methods alongside video models.Source: §2 · p. 3
• 4D generation methods [39] [43] [56] [66] [95] incorporate dynamics through multi-view and video diffusion priors.Most existing methods focus on objects because scene-level generation is difficult; WorldScore includes 4D-fy [3] because it is openly accessible.Source: §2 · p. 3

3. The WorldScore Benchmark

• WorldScore unifies 3D/4D scene generation and video generation through standardized world specifications, a curated dataset, and metrics applied to video outputs.Each next-scene task specifies spatial layout and semantic content, enabling evaluation across different methodological approaches.Source: §3 · p. 3

3.1. World Specification

• Each next-scene task specifies a current image and prompt, a next-scene prompt, and a layout containing camera matrices and a camera-movement prompt.Static tasks generate varying-length scene sequences with large camera movements to assess controllability and quality. Dynamic tasks retain the scene content but request in-scene motion under an explicitly fixed camera.Source: §3.1 · pp. 3–4

3.2. Dataset Curation

• 3000 world specifications comprise 2000 static and 1000 dynamic examples, including photorealistic scenes and stylized counterparts.Static curation retains 100 high-quality images per category across five indoor and five outdoor categories, with GPT-4o generating captions and classifications. Stylized counterparts use seven candidate styles and a style-controlled generator; dynamic curation starts from 100 manually selected images for each of five motion categories.Source: §3.2 · pp. 4–5
• 20% of static examples contain three new scenes; the remainder contain one, with layouts assigned from eight camera movements.An autoregressive LLM process generates next-scene descriptions distinct from previous scenes. Movie-style camera descriptions span all spatial directions and intra-/inter-scene movements, supporting text-to-video instructions; intra-scene layouts replace the next-scene prompt with the current-scene prompt.Source: §3.2 · p. 5

3.3. The WorldScore Metrics

• WorldScore-Static aggregates controllability and quality, while WorldScore-Dynamic additionally incorporates dynamics.Individual metrics undergo empirical linear normalization to 0–100 and arithmetic-mean aggregation; unsupported dynamics metrics receive zero for 3D scene generators. Controllability measures scale-invariant camera rotation/translation errors, object-detection success using an open-set detector, and whole-prompt content alignment using CLIPScore [22].Source: §3.3 · pp. 5, 7
• Geometric and photometric consistency separately assess scene structure and appearance using depth reprojection and optical-flow Average End-Point Error (AEPE), respectively.DROID-SLAM [72] supplies dense depth for reprojection between co-visible pixels in consecutive frames, emphasizing geometry despite appearance changes. Photometric consistency targets fine-grained texture instability that CLIP/DINO feature metrics [26] [27] fail to capture.Source: §3.3 · p. 6
• 400 participants informed selection of CLIP-IQA+ [75] with CLIP Aesthetic for subjective quality; style consistency compares first- and last-frame Gram matrices [17].The selected metric combination best matched human preferences among enumerated combinations. Style consistency uses the F-norm difference between Gram matrices within each next-scene generation task.Source: §3.3 · p. 6
• Dynamics metrics assess motion accuracy, magnitude, and smoothness using regional optical flow, consecutive-frame flow, and frame-interpolation references [93], respectively.Motion accuracy compares flow inside and outside intended motion regions to counter unintended camera movement. Magnitude assesses large-motion generation, while interpolation-based smoothness evaluates temporal jitter.Source: §3.3 · pp. 6–7

4. Results

• Human validation suggests WorldScore metrics align with human preferences and remain robust across video resolutions and aspect ratios.The benchmark evaluation covers 20 models: 13 video generation models, six 3D scene generation models, and one 4D generation model.Source: §4 · p. 7

4.1. Observations and Challenges

• 72.69 for WonderWorld and 70.40 for LucidDreamer [11] lead WorldScore-Static, exceeding the best video model, CogVideoX-I2V [88] (62.15).3D models benefit from camera controllability, larger scene spaces, and high 3D and photometric consistency, but cannot generate dynamic worlds. Extending to dynamics, 4D-fy [3] performs poorly, likely reflecting the difficulty of 4D scene generation.Source: §4.1 · pp. 7–8
• 40.22 in camera controllability makes CogVideoX-T2V [88] the best video model, yet it trails every evaluated 3D/4D model.Camera controllability is the main obstacle to strong static world generation by video models; camera conditioning [21] is suggested as a promising solution.Source: §4.1 · p. 8
• 62.15 WorldScore-Static and 59.12 WorldScore-Dynamic place CogVideoX-I2V [88] above closed-source Gen-3 and Hailuo.CogVideoX-I2V has better camera controllability but worse object controllability and content alignment than these closed-source models.Source: §4.1 · p. 8
• Larger motion often reduces smoothness and correlates weakly with motion accuracy, so magnitude alone does not ensure instruction-following motion placement.Models may instead hallucinate unintended camera motion or irrelevant motion, motivating more robust modeling across the three dynamics metrics.Source: §4.1 · p. 8
• Video models struggle with long-sequence generation and trail 3D models more in outdoor scenes than indoors.Long-sequence tasks test generation of larger worlds; Figure 7 presents WorldScore-Static performance across subdomains.Source: §4.1 · p. 8, Figure 7 · p. 8
• T2V models generally offer greater controllability and motion magnitude, whereas I2V models achieve higher quality.Empirical examination attributes this difference to T2V models generating larger camera motion while I2V models tend to retain the input viewpoint, hindering new-scene generation.Source: §4.1 · p. 8

• 3D models excel at static world generation but struggle to extend to 4D representations and incorporate dynamics.Source: §5 · p. 8
• Video models face challenges in controllability, long-sequence generation, and generating outdoor scenes.Source: §5 · p. 8
• Future research directions include bridging 3D and 4D representations, strengthening controllability mechanisms, and designing architectures for extended scene sequences.The authors envision WorldScore as a tool for measuring progress toward more capable and versatile world generation systems.Source: §5 · p. 8

A. Additional Details on World Specification

• wproc accommodates differing input requirements by formatting world specifications according to each model’s camera, text, and image control capabilities.This enables evaluation across 3D scene generation, 4D scene generation, and video generation.Source: A. Additional Details on World… · p. 13
• The world specification module defines layout using a precise camera trajectory and corresponding textual descriptions, with the trajectory serving as ground truth for camera controllability.Textual descriptions express movements such as “camera moves left.”Source: A. Additional Details on World… · p. 13
• Models supporting explicit camera control receive transformed camera poses aligned across camera types; models without that capability receive textual movement descriptions instead.This routing preserves the layout specification while accommodating different control interfaces.Source: A. Additional Details on World… · p. 13
• wproc adds camera movement text to next-scene prompts for video models without camera-matrix inputs, while leaving prompts unchanged for 3D/4D models.All evaluated 3D/4D models accept camera matrices as input.Source: A. Additional Details on World… · p. 13
• The current-scene reference image is center-cropped and resized to each model’s required resolution, providing a visual style reference and necessary input for I2V models.Model-specific resolutions are listed in Table S1.Source: A. Additional Details on World… · p. 13
• T2V models are treated as I2V models that ignore image-based control signals.This convention accommodates text-only models within the same input formulation.Source: A. Additional Details on World… · p. 13

B.1. Image Filtering

• Starting images combine existing datasets [8] [38] [57] [67] [74] with Unsplash, with filtering addressing redundancy and unsuitable viewpoints.Existing datasets can contain unusual viewpoints and narrow-angle perspectives that make images unsuitable as starting scenes.Source: §B.1 · p. 13
• Image filtering combines quality, perspective, similarity, and brightness checks with final human review.CLIP-IQA [75] and CLIP Aesthetic assess quality; Perspective Fields [28] filters extreme roll/pitch and narrow fields of view; CLIPSIM [53] removes visually similar images. Brightness thresholding excludes dark images, and manual review checks suitability.Source: §B.1 · p. 13

B.2. Stylized Image Generation

• Each photorealistic image receives a stylized counterpart generated using a text-to-image model after filtering and categorization.This extends the curated photorealistic image dataset with corresponding stylized images.Source: §B.2 · p. 13
• 7 predefined visual styles draw on art history [59] and commonly used SDXL styles.The styles are anime, cyberpunk, Chinese ink painting, ukiyo-e, impressionism, postimpressionism, and minecraft.Source: §B.2 · p. 13, Figure S2 · p. 15

B.3. Next-Scene Text Prompts Curation

• 1 to 3 prominent entities and a brief description are generated for each sequentially interconnected static scene using autoregressive GPT-4o.Entities must fit their scenes, and outputs use JSON fields named Entities and Prompt.Source: §B.3 · p. 13
• Dynamic prompts use a single GPT-4o system call to identify prominent objects capable of movement and briefly describe their motion.The model receives an image and returns JSON fields named Objects and Prompt; reasonable guesses are permitted.Source: §B.3 · pp. 13–14

C.1. Camera Controllability

• Camera controllability geometrically combines rotation and scale-invariant translation errors, using frame-wise poses estimated by DROID-SLAM [72].Rotation error measures angular deviation in degrees, while translation error uses least-squares scale alignment. The final error averages across all frames of all generated videos.Source: §C.1 · pp. 15–16

C.2. 3D Consistency

• 3D consistency measures reconstruction reprojection error using DROID-SLAM [72], whose dense bundle adjustment refines camera poses and per-pixel depths against optical flow.The dense approach uses all available points rather than selected feature matches, unlike COLMAP. Reprojection error measures Euclidean discrepancies between observed image points and projected reconstructed points over valid co-visible points.Source: §C.2 · p. 16

C.3. Photometric Consistency

• Photometric consistency measures Average End-Point Error (AEPE) after tracking center-cropped points forward and backward between consecutive frames.Higher AEPE indicates greater inconsistency, including identity shifts, texture flickering, or object disappearances. Errors are averaged across all consecutive frame pairs of all generated videos.Source: §C.3 · pp. 16–17

C.4. Subjective Quality

• Subjective quality uses the arithmetic mean of CLIP-IQA+ [75] and CLIP Aesthetic, the combination best matching human preferences in the study.The authors systematically compared quality predictors against human judgments of world-generation quality.Source: §C.4 · p. 17

C.5. Motion Accuracy; C.6. Motion Magnitude

• Motion accuracy checks whether optical flow occurs within intended object regions, while motion magnitude measures median flow to expose conservative, near-static generation.SEA-RAFT [79] estimates flow, and SAM2 [54] tracks dynamic-object masks from dataset-provided first-frame segmentations; accuracy contrasts flow inside and outside these masks. Both metrics average their frame-pair scores across all generated videos.Source: §C.6 · p. 17, §C.5 · p. 17

C.7. Motion Smoothness

• Motion smoothness averages normalized mean squared error, SSIM, and LPIPS between dropped odd-indexed frames and their interpolation-based reconstructions [93].Reference clips are the five most CLIP-text-similar OpenVid-1M [50] videos per prompt, restricted to 3–10 seconds. Bilinear reconstruction establishes empirical worst references—maximum MSE/LPIPS and minimum SSIM—while the empirical best is set to zero.Source: §C.7 · pp. 17–18, §C.8 · p. 18

C.8. Empirical Bounds

• Empirical bounds use fixed-camera and interpolated-video baselines, theoretical limits, or z-score rescaling, depending on the metric.Camera error uses zero and a fixed-camera baseline; object detection uses 0–100%; 3D, style, and photometric errors use zero and videos interpolated between randomly sampled image pairs [93]. Content alignment, subjective quality, motion accuracy, and motion magnitude use z-score rescaling to place selected models’ performance within 25–75.Source: §C.8 · p. 18

C.9. Score Normalization and Mapping

• Score normalization linearly scales raw metrics using empirical bounds, reverses lower-is-better metrics, and clips scores to [0, 1], with higher values always better.Higher-is-better metrics use (s - b_min)/(b_max - b_min); lower-is-better metrics use one minus that quantity before clipping.Source: §C.9 · p. 18

Human preference agreement score

• The probabilistic agreement score averages the fraction of participants preferring each metric’s higher-scored video, assigning 0.5 when metric scores tie.Participants made a two-alternative forced choice for each video pair.Source: D. Validation with Human Preference · p. 18

Human preference study

• 400 participants compared cross-model video pairs sampled from CogVideoX-I2V, VideoCrafter1-I2v, DynamiCrafter, WonderJourney, and InvisibleStitch using a single overall-quality question.Preliminary testing found that general raters struggled to distinguish aesthetic from perceptual quality, with highly correlated judgments motivating the single question.Source: D. Validation with Human Preference · pp. 18–19

Agreement results on subjective quality

• The arithmetic mean of CLIP-IQA+ [75] and CLIP Aesthetic achieved the highest human agreement and was selected as WorldScore’s subjective-quality metric.The supplied passage does not report numerical agreement values.Source: D. Validation with Human Preference · p. 19

Agreement results on other metrics

• Higher WorldScore metric scores consistently corresponded to higher human preference in comparisons between score buckets.The study compared buckets of 90 ± 5, 60 ± 5, and 30 ± 5; both “90 over 60” and “60 over 30” favored higher-scoring videos.Source: D. Validation with Human Preference · p. 19

Robustness against different resolutions and aspect ratios

• ≤0.83 differences across all metrics supported robustness when EasyAnimate videos were center-cropped and resized from 1344×768 to 256×256.This test changed both resolution and aspect ratio.Source: D. Validation with Human Preference · p. 19

E. Further Visualization

• 10 independent dimensions decompose world generation capability, with paired good and bad examples illustrating how metrics distinguish generated results across controllability, quality, and dynamics.Figures S3 and S4 present two generated results per metric, placing the good example on the left and the bad example on the right.Source: E. Further Visualization · p. 19, Figure S3 · p. 20, Figure S4 · p. 21
• Video generation models exhibit very low camera controllability on WorldScore-Dynamic, indicating difficulty following predefined camera trajectories.Models such as T2V-Turbo that perform well in motion magnitude tend to struggle with motion smoothness, suggesting a trade-off between large movements and temporal stability.Source: E. Further Visualization · p. 19, Figure S5 · p. 21
• 3D scene generation models such as WonderWorld achieve high camera controllability, 3D consistency, and photometric consistency on WorldScore-Static, but may struggle with subjective quality.Geometric and photometric coherence do not necessarily yield visually appealing results.Source: E. Further Visualization · p. 19, Figure S6 · p. 21