Orbia asks whether generated videos form a consistent three-dimensional world.
Orbia shifts video generator evaluation from visual appeal to coherent 3D worlds, testing 27 models on input, frame, and spatial consistency.
Key Takeaways
- Orbia is a benchmark that evaluates video generators based on whether they create a consistent 3D world, not just visually appealing frames.
- It measures three main aspects: preservation of the initial input, consistency between generated frames, and alignment with a stable 3D structure.
- The benchmark tests 27 models, including native video models, interactive world models, and hybrid approaches, using depth consistency, revisit consistency, and novel view synthesis.
- Orbia is built from real-world exploration, synthetic simulation, and object-level interaction data to assess model behavior as interactive worlds.
- Its key message is that useful video generation for simulation or exploration must remain coherent when viewpoints change, rather than only producing attractive individual frames.
Orbia, a benchmark for video generators, shifts the evaluation question from whether frames look good to whether they form one consistent 3D world. It addresses the use of video generators as interactive worlds for exploration and simulation by asking whether models preserve the initial input, keep generated frames consistent, and maintain a stable 3D structure. Orbia evaluates 27 models, including Sora, Veo 3, Kling, Wan 2.1, HunyuanVideo, Genie, Matrix-Game, Yume, WorldScore, and WorldModel, using tests for input reference, generated-frame consistency, and 3D consistency.
Why Visual Appeal Is Not Enough
Realistic and visually appealing frames do not necessarily depict a consistent 3D world. Existing benchmarks assess selected geometric properties or use vision-language models to judge memory and consistency. Orbia is built around the gap that systematic 3D evaluation remains underexplored.
Three Dimensions of the Benchmark
The benchmark organizes evaluation around three questions.
Input reference asks whether the model preserves the initial input. The metrics include reference consistency and revisit consistency.
Generated consistency asks whether the generated frames are consistent with one another. Feature matching is used to compare the relationship between frames.
3D consistency asks whether the generated frames are consistent with a 3D world. Depth consistency and novel view synthesis are included to test spatial structure.
How the Dataset Is Constructed
The dataset is constructed through three methods: real-world exploration, synthetic simulation, and object-level interaction.
What the 27 Models Cover
The evaluation covers 27 models across three categories.
Native video models include Sora, Veo 3, Kling, Wan 2.1, and HunyuanVideo.
Interactive world models include Genie, Matrix-Game, and Yume.
Hybrid approaches include WorldScore and WorldModel.
The report presents results from the 27-model evaluation, organized around input reference, generated-frame consistency, and 3D consistency.
What Follows From the Evaluation
Follow-up work can use Orbia to compare models beyond visual quality: whether they preserve reference inputs, handle revisits, produce consistent generated frames, and support depth and novel views. For readers, the practical meaning is that an interactive world should remain coherent when the viewpoint changes, not merely produce attractive frames.
Meta description: Orbia tests whether video generators create consistent 3D worlds, evaluating 27 models across input, generated frame, and 3D consistency metrics.
Tags: video generators, 3D consistency, interactive worlds, world models, benchmark
Featured image: An abstract friendly illustration of a camera path moving through a low-poly world, with translucent frames showing depth, revisit, and consistency checks; no real people, no logos, no text.

Comments (0)