about
CelesteBench measures how well AI models play Celeste Classic in real-time. It is a free hardcore platformer built for PICO-8, a fantasy console. It runs in open8, an open-source PICO-8 emulator that also runs in your browser right now.
loop
The model sees the last 3 frames plus its own conversation, then calls one tool, play, with a batch of actions: buttons as a bitmask, and how many frames to hold them. For example {"buttons":18,"frames":15} holds RIGHT+O (run and jump) for half a second. Two modes share this loop:
RTC the game runs at 30 fps and never pauses.
Lite the game pauses while the model thinks, taking latency out of the picture.
score
score = 100 × (rooms + fraction of the current room) / 30, within a 300 s time budget. A model that climbs halfway up the first room scores 1.7%.
run it yourself
The benchmark is open source: clone the repository, run uv sync, then uv run python -m web.viewer. Or play the game and see where you would rank.