South Korea Is Using AI To Resurrect a Dead Superstar's Voice

The 1996 demise of popular South Korean folk singer Kim Kwang-seok has been shrouded in mystery, leaving fans to devise their own theories, while hoping to hear his voice one last time. Thanks to new developments in the field of AI, fans will now get to hear his voice on national television for the first time in 25 years, singing new material. 

Korean National broadcaster SBS plans to use artificial intelligence to revive Kwang-seok’s voice on a new programme, "Competition of the Century: AI vs Human," to air later this week. The announcement has sparked a wave of excitement among the singer’s fans, who still participate in an annual gathering in a street near his childhood home in the city of Daegu.

A one-minute promotional clip of Kim singing "I Miss You," a ballad released by Kim Bum-soo, another famous Korean singer, in 2002, has fetched over 150,000 views on YouTube since December 2020. Another video that shows behind-the-scenes footage from the production of the episode has garnered over 750,000 views in just three weeks since its release.

“The recovered voice sounds very much like him, as if Kim recorded it alive,” Kim Jou-yeon, a Kim fan for 30 years, told CNN. 

This is not the first time a South Korean celebrity has been brought to life through AI. In December, music channel Mnet aired One More Time, a show that used AI and holograms of other late artists to pay tribute to their work. On New Year’s Eve, famous boyband BTS also performed with an AI-generated likeness of singer Shin Hae-chul, who died after surgery in 2014.

While these programmes continue to strike a chord with fans of late singers and tech enthusiasts alike, they have also given rise to ethical concerns attached to resurrecting the voices of the dead. The creation of new material in dead people’s voices through AI also raises copyright issues, since it’s not clear if the owner is the creator of the AI or the AI system itself.

SBS Producer Nam Sang-moon has said that the idea for a human vs AI competition struck him when he watched world champion Lee See-dol play South Korean AI programme HanDol at the ancient strategy game of Go in 2019. In an impressive and unexpected feat, Lee managed to win one of their three matches, when only a month ago, he had declared his retirement from the game, citing AI as an undefeatable competitor.

This game had reminded producer Sang-moon of See-dol's earlier match against AlphaGo, an AI programme developed by Google DeepMind, in 2016. That time, AlphaGo won four out of five games, making Lee admit he had "misjudged" the machine's capabilities. 

He then began pulling together the six episode competitive series, featuring AI performances from the late Kim Kwang-seok, with help from Supertone, a South Korean startup founded in 2020 that provides AI audio solutions for content creators. 

“For example, BTS is really busy these days, and it'd be unfortunate if they can't participate in content due to lack of time. So, if BTS uses our technology when making games or audiobooks or dubbing an animation, for instance, they wouldn't necessarily have to record in person,” co-founder and Chief Operating Officer Choi Hee-doo said. 

Supertone's Singing Voice Synthesis (SVS) technology learns voices by listening to multiple songs with corresponding lyrics and notes. Before learning Kim Kwang-seok’s voice in ten of his songs including hits like “A Letter From a Private,” “Song of My Life” and “In the Wilderness,” the system had to learn 100 songs by 20 other singers. This ensured that the AI system now knows Kim’s unique voice well enough to imitate his style and pronunciation. 

In the upcoming SBS show, the AI Kwang-seok will be singing a duet alongside a human singer. Ock Joo-hyun, the former lead singer of girl band Fin.K.L, will be taking on the AI machine, which has also been tasked with learning her voice. 

Even outside of South Korea, there have been several attempts to resurrect deceased singers—from Elvis to Tupac through holograms, which have been both well-received and controversial. Hologram concerts have also been around for a while, but have struggled to create the same connect between fans and the artist that live performances are known for, in the absence of essential concert elements like audience interaction.

The reason singers are picked to be replicated over other public figures known for their speeches and ideology, such as Winston Churchill or Martin Luther King Jr., is because AI can only imitate mannerisms and voice. What it cannot do is predict what a person would say in a given situation, regardless of the amount of data it has. 

Recently, however, singer Kanye West tried to achieve this feat when he gifted wife Kim Kardashian a hologram of her late father Robert Kardashian, which (who?) quickly became meme material for crowning West as “the most, most, most, most, most genius man in the whole world”.

While reviving people’s voices might simply be intended as a way to entertain and enthral audiences, such attempts are fraught with technological concerns that must be addressed with stricter guidelines and regulations. In the past, fake voices have been used for misinformation campaigns and fraud. In 2019, a U.K. based executive was scammed out of $243,000 by using AI voice technology that made him believe he was on the phone with his boss. Internet security experts at Symantec have reported at least two more cases of using audio deepfakes to scam people of big bucks since then.

These are not the only issues that complicate the matter though. Resurrecting singers’ voices using AI also raises the question of ownership. Can this style of AI-generated music be copyrighted? And if yes, then by who? Who holds the ownership of the voice being replicated?

In the Kim Kwang-seok case, producer Sang-moon claims SBS had sought and received the consent of Kim's family to reproduce his voice before proceeding with the show. Just like with the other cast members, Kwang-seok’s family were paid a one-time fee for featuring his voice on the show. 

While this is major news for Kwang-seok’s fans, they must be alert enough to catch the show while it airs, because neither Supertone nor SBS currently have any plans to release his new material as a single.

Follow Snigdha on Twitter.