Claude AI Now Plays Pokémon On Twitch

Modern AI technology has a wide range of applications, anything from enhancing videos, to research, and now it can also be used to play the classic Gameboy title, Pokémon Red, live on Twitch, on its own. It’s not quite as advanced as the AI seen in movies, but it’s still impressive nonetheless.

ClaudePlaysPokemon is a passion project run by members of Anthropic, an AI safety and research company, and Claude playing Pokémon live online is part of testing its capabilities. 

“You can watch its thought process in real-time, showing how an AI approaches problem-solving in an interactive environment,” the Twitch channel states. “Unlike traditional game bots that follow predetermined rules, Claude can see what’s happening, understand the game state, and make decisions, similar to how a human player would.”

How it Works

The live stream shows the game on one side, and Claude’s actions on the other. Viewers can see in real-time how Claude reasons its way through the game, but as for how it does this, it’s quite involved.

Claude can control the game by pressing virtual buttons, and the AI navigates by analyzing screenshots. As such, it can navigate to specific coordinates on the screen and then update its knowledge base with any new information it discovers. This adds to its reasoning ability and helps to prevent it from getting confused by what’s happening.  

A Few Kinks in The System

All of this is what helps Claude carry out specific and complicated actions, such as turn-based combat against enemies. Viewers have already expressed surprise at Claude’s ability to understand that initiating certain attack types can lead to, for example, bonus attacks or that it should change its tactics.

However, ClaudePlaysPokemon does make it clear that what people see is still very much a learning process.

“Claude’s ability to understand Game Boy screens isn’t great—you’ll often see it misinterpret what it’s seeing. Claude can be a little silly sometimes – tune in and enjoy the show!”