Games such as Dance Dance Revolution are extremely hard to program. It takes a lot of time to develop and implement a single level, from deciding the music and foot placement to the actual programming of the chosen steps into the game.
Most of the levels were designed by the game developers, sometimes even by the fans using StepMania, a port for DDR. A new way of designing the step charts is through the training and use of intelligent neural networks.
Neural networks are capable of analyzing and identifying patterns in data to create outputs of similar nature. The neural networks work in three steps. First, they convert the songs into graphical representations of audio frequency components called spectrograms. This is used to identify features of the music such as pitch and rhythm.
Secondly, an algorithm is used to divide the song into 10-millisecond samples. This is done to decide whether that small section of the audio contains a step or not. Lastly, a selection algorithm is used to map the choices for steps onto moves completing the DDR step chart.
A trio of computer engineers from the University of California came up with the idea when one of them remembered having data from the game StepMania. He had data from two main choreographers, which totaled to about 35 hours of music and 350,000 steps.
The end result is that the synthetically generated steps are close to the actual man-made steps and are still playable by humans, though experts will still tell the difference. A choreographer, whose charts were used to train the neural networks noted that it was easy to tell the output was made by a computer. He said there were certain creative techniques the Artificial Intelligence could either not replicate or didn’t learn, to begin with.