‘Pokémon Go’ Players Are Training AI Models To See The World

Millions of Pokémon Go players who thought they were just playing a fun game might actually be helping to train AI models that can map the physical world.

Niantic, the company behind Pokémon Go and other mobile games, released a blog post earlier this month, describing how it’s using 3D data collected in its games to create a “large geospatial model.”

In the same way that large language models, such as those used to power ChatGPT, can accurately predict which words should come next in a sentence, so large geospatial models will be able to infer what buildings and other physical objects should look like, even if they only have photography of part of them.

“As humans, we have ‘spatial understanding’ that means we can fill in these details based on countless similar scenes we’ve encountered before,” reads the blog post, written by Niantic’s staff scientist Eric Brachmann and chief scientist Victor Adrian Prisacariu.

“But for machines, this task is extraordinarily difficult. Even the most advanced AI models today struggle to visualize and infer missing parts of a scene, or to imagine a place from a new angle.”

Pokémon’s Visual Positioning System

The large geospatial model is taking advantage of the Visual Positioning System that Niantic has built into games such as Pokémon Go. This can use a single image taken by a smartphone camera to determine its position and orientation with centimeter-level accuracy, according to the company.

In Pokémon Go, players use VPS for features such as Pokémon Playgrounds, where players can place virtual Pokémon at a specific location for others to find.

This creates a vast trove of 3D data for the company to work with. “Today we have 10 million scanned locations around the world, and over 1 million of those are activated and available for use with our VPS service,” Niantic’s blog claims. “We receive about 1 million fresh scans each week, each containing hundreds of discrete images.”

The company explains how the large geospatial model could use such data to effectively fill in missing detail on mapped buildings. It cites the example of a church, where you currently only have data showing what the front entrance of the building looks like.

“An LGM would be able to internalize the concept of a church, and, furthermore, how these buildings are commonly structured,” the blog claims. “Even if, for a specific location, we have only mapped the entrance of a church, an LGM would be able to make an intelligent guess about what the back of the building looks like, based on thousands of churches it has seen before.”

Potential Uses

Niantic claims data large geospatial models could be used for a wide variety of purposes, including gaming, spatial planning and logistics.

However, critics fear that Pokémon Go players may have unwittingly contributed to something altogether more sinister, suggesting it could be used by the military. “It’s so incredibly 2020s coded that Pokémon Go is being used to build an AI system which will almost inevitably end up being used by automated weapons systems to kill people,” tweeted Elise Thomas, a senior intelligence analyst at the Institute for Strategic Dialogue, as first reported by Garbage Day.