This is a dancing game consisting of a screen and a dance platform that players control with their feet. The platform has four pads, which players must touch to music in the order specified by a chart on the screen. So players must dance to the music in the way the game demands.
The game also allows players to design and distribute their own dances. Over the years, people have created enormous databases of dances for a huge range of popular songs.
That gave Chris Donahue and pals, at the University of California, San Diego, an idea. Why not use this huge database to train a deep-learning machine to create dances of its own?
Today, they show how they have done just that. Their system—called Dance Dance Convolution—takes as an input the raw audio files of pop songs and produces dance routines as an output. The result is a machine that can choreograph music.
The game itself is straightforward in principle. As the music plays, the player touches the pads on the dance platform in the order shown on the screen. Each pad can be in one of four states: on, off, hold (or freeze), and release. Because the four pads can be activated or released independently, there are 256 possible step combinations at any instant.
Of course, the dances become progressively harder, with most songs having dances with five levels of difficulty. The difficulty is determined by the speed of the rhythmic subdivisions. Beginner-level games have steps on quarter and eighth notes, but higher difficulty dances have 16th note steps and some patterns involving 12th and 24th notes.
There are also other informal rules for the creation of dance charts. “Chart authors strive to avoid patterns that would compel a player to face away from the screen,” say Donahue and co. The result is dances with a wide variety of rich structures.
The task of automating the creation of dance charts is by no means simple. Donahue and co divide it into two parts. The first is deciding when to place steps, and…