Youtuber and programmer Harrison Kensley, or “Sentdex,” recently unveiled GAN Theft Auto, a neural network-generated version of Grand Theft Auto 5.Kensley created the game in collaboration with Daniel Kukiela using a fork of NVIDIA's GameGAN research. GANs, or generative adversarial networks, are a type of unsupervised machine learning that takes specific real-world inputs (in this case, a stretch of highway in GTA5 and player input) and tries to replicate them. Often, two (or more) neural networks are pitted against each other, and the "better" outputs are kept, allowing the AI to "learn" from each other. In the video unveiling the game, Kensley’s seen driving a black car along a fuzzy, dream-like rendering of a highway. Ripples move through the image as the neural network creates the next frame and creates its own rules. “We are playing entirely within a neural network’s representation of Grand Theft Auto 5,” Kensley says in the video.Kensley and Kukiela first tested the process on a minigame they created called Vroom that showed a simple race car traveling on a winding track. After rounds of training, the neural network was able to create clearer visuals and also emulate the constant movement and boundaries of the game.NVIDIA then sent Kensley a DGX Station A-100, a server-grade AI platform or data center designed to be more workplace friendly, which led them to start modeling Grand Theft Auto 5. They created 12 rules-based AIs to drive around the same highway stretch in GTA 5, gathering different types of data, visuals and player input.The earliest model was able to create a pixelated rendering of the highway that lacked physical boundaries but included minute details such as the shadow of the car and the reflection of the windshield.In later models, the neural network is able to set boundaries so the car drives into a wall rather than over the edge of the highway. “There are no rules written here by us or the original Grand Theft Auto engine. The neural network is controlling all of this,” Kensley says.In NVIDIA’s original GameGAN research, it successfully modeled a game of Pac-man. The neural network was able to do everything from turning the ghosts blue when Pacman ate a capsule to also recovering the generated layout when Pacman returned to other parts of the maze. But NVIDIA also found that it could extricate the static and dynamic components of the game from each other. Meaning, the AI could add completely new components to a game–such as playing with Mario in the maze instead of Pacman. In the video, Kensley speculates just how much more of the game could be generated by the neural network. “This is definitely Grand Theft Auto 5, only it’s completely generated by an AI and we get to play within it. And that is just incredibly awesome to me,” he says.
Advertisement