How to turn a game into an RL environment

The technical intuition, worked end to end on one example: curating a dataset that makes the task learnable, designing the environment, shipping it with OpenEnv so other people can run it, and training a Qwen3.5-4B against it with TRL until it outscored gpt-5.4-mini and claude-haiku-4.5 at GeoGuessr.

Every figure is built from the published records, and everything it cites is on the Hub:

• geoguesser-env is the environment, playable in a browser
• geoguesser-tasks holds both task splits
• geoguesser-panos holds the 22 GB of imagery
• geoguesser-trackio has all four training runs
• the collection gathers the lot

Source, environment, eval harness and training script live in HuggingEnvs under 03-geoguesser/, with the article itself under content/articles/geoguesser/.