BRISTOL, United Kingdom — Video game culture has come a long way since the days of Pong and Pac-Man. Nowadays, many gamers engage in “speed runs” aimed at completing a level or entire game as fast as possible. Many speedrunners take advantage of glitches (also known as exploits) in games like Super Mario Bros. to finish all of the levels faster. Now, researchers from the University of Bristol say that these gaming hacks may actually help tech experts better understand problems plaguing modern computer software.
Students at UB analyzed four classic Super Mario games, as well as 237 known glitches within those games, classifying a variety of weaknesses along the way. All in all, this project explored if these vulnerabilities are the same as the bugs exploited in more conventional software.
Nintendo’s Super Mario series is considered one of the quintessential video game franchises of all time. To understand the sorts of glitches speedrunners typically exploit, study authors examined four of the earliest Mario platforming games: Super Mario Bros (1985), Super Mario Bros. 3 (1988), Super Mario World (1990), and Super Mario 64 (1996).
While all of those games are on the older side, they are all still competitively played by speedrunners, with new records frequently reported in gaming news publications. Importantly, those games are also well understood, having been studied by speedrunners for decades. This ensured there would be large numbers of well-researched glitches for analysis.
The current world record time for beating Super Mario World is an astounding 41 seconds. Researchers set out to better understand all 237 known glitches within the games and classified a variety of weaknesses along the way – all in an effort to potentially help software engineers make more robust applications.
In the universe of the games, Mario attempts to rescue Princess Peach by jumping through an obstacle course made up of various platforms to reach a goal, avoiding various enemies or defeating them by jumping on their heads along the way. Players collect power-ups along the way, unlocking special abilities as well as coins to increase their score. The Mario series is one of Nintendo’s flagship products and is considered one of the most influential video game series of all time.
“Many early video games, such as the Super Mario games we have examined, were written for consoles that differ from the more uniform PC-like hardware of modern gaming systems. Constraints stemming from the hardware, such as limited memory and buses, meant that aggressive optimization and tricks were required to make games run well,” says Dr. Joseph Hallett from Bristol’s School of Computer Science in a media release.
“Many of these techniques (for example, the NES’s memory mapping) are niche and can lead to bugs, by being so different to how many programmers usually expect things to work.”
“Programming for these systems is closer to embedded development than most modern software, as it requires working around the limits of the hardware to create games. Despite the challenges of programming these systems, new games are still released and retro-inspired.”
Categorizing bugs in software helps developers understand similar problems and bugs. The Common Weakness Enumeration (CWE) is a category system pertaining to hardware and software weaknesses and vulnerabilities. Researchers identified seven new categories of weakness that had previously gone unspecified.
“We found that some of the glitches speed runners use don’t have neat categorizations in existing software defect taxonomies and that there may be new kinds of bugs to look for in more general software,” Dr. Hallett adds.
Study authors performed a thematic analysis using a codebook of existing software weaknesses (CWE), which is a qualitative research method used to help categorize complex phenomena.
“The cool bit of this research is that academia is starting to treat and appreciate the work speedrunners do and study something that hasn’t really been treated seriously before. By studying speedrunners’ glitches we can better understand how they do it and whether the bugs they use are the same ones other software gets hacked with,” Dr. Hallett concludes. “It turns out the speedrunners have some tricks that we didn’t know about before.”
Moving forward, the research team is now studying Pokémon video games.
This research was presented at the Workshop on Games and Software Engineering (GAS) at the International Conference on Software Engineering (ICSE).