Nobody checked the magic prompt

In April 2025 a long, lovingly iterated prompt went around claiming to turn OpenAI's o3 into a savant at GeoGuessr — guessing a location from a single photo. People marveled at it for thirteen months. Nobody ran the obvious test.

"Models will happily make up stories for you about their own reasoning processes, and will almost always say 'yes, that helped.'" — Sean Goedecke

Sean Goedecke did. He fed roughly 200 photos to o3 twice: once with the famous engineered prompt, once with a bare 'where was this taken?' He scored each guess by how far off it was. The default won on every measure — closer median guess, closer average, more hits inside 100 kilometers. The fancy prompt didn't just fail to help; it slightly hurt. The whole experiment cost about fifteen dollars and an afternoon.

The mechanism is the quietly damning part. When a model is already good at something, extra instructions ride along without adding signal — and the usual way people 'refine' a prompt is to ask the model whether a change helped. It almost always says yes. A result everyone admired turned out to rest on that flattery, never on a measurement.

The case isn't airtight, and Goedecke says so: no significance test, modest gaps, and the photos are public-domain images o3 may have partly memorized rather than reasoned about. But the real finding sits above the numbers. It took one person, one afternoon, and pocket change to check a celebrated AI claim that a year of confident writing never bothered to.