This might be the wildest thing I have seen in 2017 so far, holy cow. Watch in wonder as someone uses Pokémon Yellow to push the boundaries of what a Game Boy is capable of.
Earlier this month, MrWint uploaded an incredible tool-assisted speedrun where he uses “arbitrary code execution” (ACE) to launch the visuals of multiple games within Pokémon Yellow via a Game Boy emulator. ACE, as some of you know, is well-known method where players manipulate a game’s memory through simple actions to alter the in-game rules on the fly. That’s what is happening at the start of this footage, when the player appears to be throwing items away.
Theoretically, ACE allows you to run pretty much anything once you’ve got it going, though the most well-known use might be forcing a game to immediately skip to the end. Technically speaking, you can actually use ACE to do so much more, and that’s exactly what MrWint tries to accomplish through this run.
The first thing you see here is ACE being used to “launch” Pokémon Gold, a game released after Pokémon Yellow. MrWint is just getting started, though. Within Pokémon Gold, he speaks to an NPC, who asks the player to look at their favorite game. That’s when Tetris starts, and the speedrun begins to feel like inception.
Tetris is then beaten, and the player walks outside...and the freakin’ game then turns into ZELDA. What the hell? Before you can wrap your brain around that, Link walks into a house, and the game goes back to Pokémon...only to walk up the stairs and have the whole thing change to a different generation of Pokémon once more. From there, the player uses the in-game NES to go into Super Mario Bros.. Mario beats 1-1, after which the player returns to Pokémon Yellow briefly. He walks down the stairs, only to appear at the end of the game, where he speaks to Professor Oak. That’s when “Still Alive” from Portal starts playing, I shit you not.
As if anticipating the viewer’s reaction, the song fades into a Spongebob clip where Patrick the star goes, “how does he dooooo that???” Then the speedrun ends and the Pokémon Yellow credits roll. Phew.
Reminder: this was all within Pokémon Yellow, hence why the final portion goes back to the original game. So how in the flying fuck does MrWint even do this? There’s a long, technical post over at TASVideos where he breaks it down, but here’s an important bit:
I don’t need to run all of the actual code, I just need to run equivalent code that produces the same audio-visual effect as the original. At first I thought about streamlining the original code by cutting not needed code paths and priming it so that it produced the predetermined results I wanted (basically pre-computing the emulation and only running the resulting instructions), but I came up with an even more radical idea: I realized that all the instructions that really mattered are those that put tiles on the screen or played some sounds. So all I need to do is emulate the actual audio-visual output of the game with the right timing, without any internal game state.
This realization was the key to this run, as it opened many more possibilities: The source of the A/V doesn’t need to be another game. It could be a hack of a game. Or the mash-up of different games. Or from a different system. Or literally arbitrary A/V. This made this run become not about running a game in another game, but more about pushing the limits of the Gameboy hardware and see what is possible.
In short, MrWint forces Pokémon Yellow to play arbitrary Game Boy footage in real time. Granted, this is more than just some visual trickery. Through this experiment, MrWint also tests the limit of the Game Boy’s audio, especially once the Portal song goes off:
Sound in a [Game Boy] turns out to be very limited in its abilities. It has 4 sound generating channels that can be connected to two output terminals. The first two channels generate square waves of different frequencies and amplitudes, with limited control over frequency and amplitude over time, and the last channel produces static noise.
Only the third channel is interesting, as it allows arbitrary wave patterns to be played. However, the RAM that holds the wave pattern only contains 32 samples that are repeated over and over, with only 4 bits per sample (i.e. 16 different possible values). It was clearly not designed for complex sounds like voice, but rather as an alternative way to creating waves with unusual shapes. You can hear this clearly in the title screen of Pokémon Yellow, with the very crude sound they achieved by overlaying multiple waves: You can hear the words, but it’s not pleasant.
However, you can use the third channel to play longer pieces of arbitrary audio, by managing to update the wave RAM while the sound is playing. This of course requires perfect precision when to update them, to ensure they are played once and only once.
The Spongebob bit, meanwhile, was MrWint pushing how far the Game Boy can render colors:
Part of it was that I wanted to show off so-called HiColor graphics. The Gameboy only has space to store 8 palettes each background tile can choose from, with 4 colors each, so the maximum amount of colors you can use on each frame is usually 32, and each 8x8 tile area can only use 4 of them at a time (plus some extra colors for the sprites which draw from different palettes, but they’re not useful for this purpose). The so-called HiColor technique allows you to use significantly more colors in an image, by changing the palettes for each rendered line. This way, each line could use its own colors, even within the same 8x8 tile. This technique was not originally intended in the Gameboy’s design, but it was actually used in some commercial Gameboy games.
I think the YouTube commenters sum up this whole shebang accurately:
You can read the full TASVideos log breaking down how this run was accomplished here, and check out the source code for the tools and programs utilized in this run here.