World Labs’ AI can generate interactive 3D scenes from a single photo | TechCrunch

World Labs, the startup founded by AI pioneer Fei-Fei Li, has unveiled its first project: an AI system that can generate video game-like, 3D scenes from a single image.

Lots of AI systems can turn a photo into 3D models and environments. But World Labs’ scenes are unique in that they’re interactive — and modifiable.

“[Our tech] lets you step into any image and explore it in 3D,” World Labs wrote in a blog post. “Beyond the input image, all is generated.”

The AI-generated scenes, which anyone with a keyboard and mouse can explore on a demo on World Labs’ website, look impressive — if a bit cartoonish. They’re rendered live in the browser, and have a controllable camera with an adjustable simulated depth of field (DoF). The stronger the DoF effect, the blurrier background objects appear.

World Labs’ system is a part of an emerging category of AI called “world models.” Many of these models can simulate games and 3D environments — but with artifacting and consistency issues. For example, startup Decart’s Minecraft-simulating world model, Oasis, has a low resolution and quickly “forgets” the layout of levels.

In contrast, World Labs’ approach ensures that scenes remain the same once they’re generated, and that they obey the basic laws of physics, meaning they have a sense of solidity and depth.

World Labs’ system can also apply interactive effects and animations to scenes, like changing the color of objects and dynamically lighting backdrops.

“Most generative AI tools make 2D content like images or videos,” World Labs wrote. “Generating in 3D instead improves control and consistency. This will change how we make movies, games, simulators, and other digital manifestations of our physical world.”

Now, there’s certainly room for improvement. World Labs’ scenes aren’t fully explorable — your movements are limited to a small area. (Try moving outside it, and you’ll hit a boundary.) And there’s the occasional rendering error, for example objects that blend together in unnatural ways.

But World Labs says that it’s only an “early preview.”

“We are hard at work improving the size and fidelity of our generated worlds, and experimenting with new ways for users to interact with them,” the startup wrote in the blog.

World Labs, which launched earlier this year, has raised $230 million in venture capital from investors including Andreessen Horowitz (a16z), Ashton Kutcher, Intel Capital, AMD Ventures, and Eric Schmidt. Valued at over $1 billion, the company hopes to have its first product ready in 2025.

Beyond interactive scenes, World Labs plans to build tools that might be useful to professionals like artists, designers, developers, filmmakers, and engineers. It’s targeting customers ranging from video game developers to movie studios.

“We already have the ability to create virtual, interactive worlds, but it costs hundreds and hundreds of millions of dollars and a ton of development time,” World Labs co-founder Justin Johnson said on a recent episode of the a16z podcast. “[World models] will let you not just get an image or a clip out, but a fully simulated, vibrant, and interactive 3D world.”