This story is over 5 years old.


This Marimba-Playing Robot Composes Its Own Music

Researchers fed the AI nearly 5,000 songs, from Lady Gaga to Beethoven.

Shimon, a four-armed robot marimba player, is now capable of writing its own songs.

At last year's Moogfest, a four-day music and technology conference in North Carolina, Shimon showed off its ability to improvise and play with human musicians. The robot has now graduated to becoming a full-fledged composer.

A product of more than 12 years of work by researchers at the Georgia Institute of Technology, Shimon is a prime example of the nascent creative capabilities of robots.


To teach Shimon to make music, Ph.D student Mason Bretan and his team fed the bot's neural network more than 5,000 songs—including selections from Beethoven, the Beatles, Lady Gaga, and Miles Davis.

"This is a leap in Shimon's musical quality because it's using deep learning to create a more structured and coherent composition," Gil Weinberg, the bot's original creator and the director of Georgia Tech's Center for Music Technology, said in a statement provided to Motherboard.

Shimon is fairly independent, but does need the first four measures of a song before it can take off on its own.

"Once Shimon learns the four measures we provide, it creates its own sequence of concepts and composes its own piece," Bretan said. "Shimon's compositions represent how music sounds and looks when a robot uses deep neural networks to learn everything it knows about music from millions of human-made segments."

Shimon isn't the only musical robot that Weinberg has created. He and his team also developed Shimi, a "robotic musical companion." The dancing friend can shimmy to the beat of any song you choose.

This isn't the first project of its kind. IBM's Watson supercomputer is also capable of writing original music, and various other researchers and hobbyists have made artificial intelligence that can write music. What makes Shimon different though is that the robot can actually play a musical instrument.

Shimon's first live performance as a composer will be at the Aspen Ideas Festival at the end of this month.