A groundbreaking research project from Virtual Protocols has demonstrated how generative AI can simulate a playable version of classic video games like Super Mario Bros. Using its newly developed model, MarioVGG, the team has showcased how AI can generate a version of a 1985 Nintendo classic from simple text inputs such as “run” or “jump.”
The project which debuted on Wednesday, highlights the potential for AI to revolutionize game development. According to a research paper, MarioVGG uses text-to-video diffusion technology to generate consistent, controllable game frames that mimic the original game’s mechanics.
AI-Powered Game Generation
MarioVGG was created by training an AI model on over 737,000 frames from a Super Mario Bros. session, analyzing actions such as Mario’s running and jumping. AI model can now generate similar game frames based on a player’s text commands, creating a sequence that mirrors the action.
“Trained on just a single consumer-grade GPU (an Nvidia RTX 4090), our text-to-video diffusion model can generate controllable game frames that are consistent and follow game mechanics,” says the paper.
While the model is not yet fully playable and still faces some lag—taking approximately four seconds to generate a six-frame video sequence—it provides an impressive simulation of the original Super Mario Bros. gameplay.
Simulating Game Mechanics and Levels
MarioVGG not only replicates Mario’s actions but also accurately simulates game mechanics, such as gravity effects. According to researchers, the model can depict Mario falling off platforms and interacting with the environment as he runs.
Moreover, MarioVGG can generate new and unique levels, with the first frame being only one drawn directly from a true game state. However, generated levels and objects remain procedurally created and are not yet controllable by the player.
Challenges and Limitations
While Mario VGG successfully simulates many of the 2D platformer’s mechanics, it does have limitations. For instance, the model occasionally fails to follow the game’s rules and logic. One issue includes AI continuing to render frames even after Mario dies, without reverting to last checkpoint, which would normally occur in original game.
Furthermore, the frame resolution remains lower than the original game’s, with each frame being generated at 64-by-48 or 128-by-96 resolution, compared to the 256-by-224 resolution of the 1985 version.
Despite the challenges, research offers a glimpse into how AI-enabled video generation might one day transform the gaming industry, providing developers with new tools to create interactive worlds faster and more efficiently.
Future of AI in Game Development
Virtual Protocols’ experiment with MarioVGG is part of a larger trend of AI-powered game development. Potential for generative AI to create game assets, levels, and even entire gameplay experiences could revolutionize how video games are made, offering faster production cycles and more customizable experiences for players.
As AI continues to evolve, MarioVGG hints at exciting new possibilities for developers and gamers alike. However, many AI-generated game elements are still in the early stages of development. Future iterations could lead to fully playable AI-generated games that adhere closely to original mechanics and designs.
