In a groundbreaking project named Predicting Image Geolocations (PIGEON), three Stanford graduate students have demonstrated the remarkable geolocation capabilities of artificial intelligence (AI). The project, initially designed to identify locations on Google Street View, showcased AI’s potential to accurately determine the origin of personal photos it had never encountered before.
Inspired by their shared enthusiasm for the online game GeoGuessr, the trio—Michal Skreta, Silas Alberti, and Lukas Haas—sought to develop an AI player capable of outperforming human participants in geolocating photos. GeoGuessr, with its 50 million players worldwide, became the playground for their ambitious experiment.
The students built upon the existing CLIP neural network, developed by OpenAI, to analyze images by reading accompanying text. Their training dataset comprised approximately 500,000 images from Google Street View. Despite the relatively modest size of the dataset, the resulting PIGEON system exhibited impressive performance, correctly identifying the country in 95% of cases and pinpointing locations within approximately 25 miles of the actual site.
Silas Alberti explained, “We created our own dataset of around 500,000 street view images. That’s actually not that much data, [and] we were able to get quite spectacular performance.”
To enhance the system, the students integrated features that allowed the AI to classify images based on their global position. PIGEON’s capabilities extend beyond mere identification, making it a formidable competitor even against seasoned human geoguessers. In a head-to-head competition, PIGEON emerged victorious against Trevor Rainbolt, a renowned geolocation expert.
While the project highlights the potential for positive applications, such as aiding in identifying locations in old family photos or assisting field biologists in conducting rapid surveys, concerns about privacy have been raised. Jay Stanley, a senior policy analyst at the American Civil Liberties Union, emphasized the sensitivity of location information from a privacy standpoint. He cautioned against the potential misuse of this technology for government surveillance, corporate tracking, or even stalking.
The PIGEON project, born out of Stanford’s Computer Science 330 class on Deep Multi-task and Meta Learning, has not only showcased the AI’s prowess in geolocation but has also opened a dialogue on the ethical implications of such capabilities. As AI continues to evolve, striking a balance between innovation and safeguarding individual privacy remains a critical consideration.
In the words of Silas Alberti, “We weren’t the first AI that played against Rainbolt. We’re just the first AI that won against Rainbolt.” The PIGEON project stands as a testament to the evolving landscape of AI applications and the need for responsible development in the face of unprecedented technological advancements.
