BlenderWorld is a benchmark for evaluating visual agents that create three-dimensional content from real-world references.
BlenderWorld evaluates visual agents on 223 real 3D tasks using editable projects, fine-grained criteria, and model judges across 13 frontier models.
Key Takeaways
- BlenderWorld is a new benchmark from the Qwen team for evaluating visual agents on realistic, open-ended 3D creation tasks.
- It includes 223 tasks across 9 domains and 22 subcategories, requiring agents to produce complete, editable Blender projects rather than just renders.
- Evaluation is highly detailed, using 2,651 task-specific criteria across 13 aspects, with VLM-as-Judge and Agent-as-Judge assessing visual and project-level quality.
- Testing 13 frontier models showed progress, but even the leading model, GPT-6-Astra, fully satisfied every requirement on only 42.2% of tasks.
- The benchmark highlights ongoing weaknesses in multimodal reference interpretation, fine-grained requirement compliance, and spatial or physical reasoning.
The Qwen team introduced BlenderWorld, a benchmark that gives researchers and developers a systematic way to test whether visual agents can handle real-world 3D creation. Instead of only checking whether an agent produces a website, poster, or slide deck, it asks whether the agent can build complete, editable Blender projects from realistic instructions.
Why 3D creation is harder than websites or slides
Visual agents can already generate practical outputs such as websites, posters, and slide decks. Creating 3D content is a different challenge because it requires spatial reasoning, tool use, and iterative visual feedback. Existing benchmarks have not covered realistic, open-ended 3D tasks broadly enough to show where those weaknesses appear. BlenderWorld is presented as the first comprehensive benchmark for systematically evaluating visual agents on real-world 3D creation tasks.
What BlenderWorld measures
The benchmark includes 223 realistic tasks across 9 domains and 22 subcategories. The domains cover areas such as architecture, product design, nature, character animation, and scientific visualization. Tasks can accept different kinds of input, including text, images, videos, and structured specification files. The expected deliverable is not just a render that looks attractive. It is a complete Blender project that can be opened and edited afterward.
How scoring goes beyond a pretty render
BlenderWorld breaks evaluation into 2,651 task-specific, fine-grained criteria. These criteria cover 13 aspects, including geometry, spatial layout, materials, and functionality. To score visual quality, an agent automatically selects and renders 8 key views. A vision-language model judge, referred to as VLM-as-Judge, evaluates the rendered output. An Agent-as-Judge also inspects the underlying project structure. Overall quality is assessed across 8 dimensions, including realism and physical plausibility.
What the 13 models revealed
The Qwen team evaluated 13 frontier models on the benchmark. GPT-6-Astra leads with a score of 79.9. The result shows progress, but it also shows a gap: even the strongest model satisfies every fine-grained requirement on only 42.2% of tasks. That distinction matters because a high average score can still hide missed details. The results are promising, with substantial room for improvement in interpreting multimodal references, meeting detailed requirements, and handling physical and spatial relationships.
Where to explore the results
BlenderWorld also includes an interactive 3D gallery where users can explore, rotate, and compare creations from different models directly in a browser. The project is documented in a paper at https://arxiv.org/abs/2610.23986, and the project site is available at https://blenderworld.github.io/. The code and dataset are publicly available at https://github.com/QwenLM/BlenderWorld. The team asks readers who find the work useful to consider starring the repository on GitHub.
Meta description: BlenderWorld benchmarks visual agents on 223 three-dimensional tasks across 9 domains, testing 13 frontier models with editable Blender project evaluation.
Tags: BlenderWorld, visual agents, three-dimensional benchmark, GPT-6-Astra, Qwen team
Featured image: Abstract friendly illustration of a three-dimensional design workspace with eight rotating model views, no real people, no logos, no text.

Comments (0)