Video conferencing apps like Zoom and Skype are usually boring and often frustrating. With more people than ever using this software to work from home, users are finding new ways to spice up endless remote meetings and group hangs by looping videos of themselves looking engaged, adding wacky backgrounds, and now, using deepfake filters for impersonating celebrities when you're tired of your own face staring back at you in the front-facing camera window.
Avatarify is a program that superimposes someone else's face onto yours in real-time, during video meetings. The code is available on Github for anyone to use.
Programmer Ali Aliev used the open-source code from the "First Order Motion Model for Image Animation," published on the arxiv preprint server earlier this year, to build Avatarify. First Order Motion, developed by researchers at the University of Trento in Italy as well as Snap, Inc., drives a photo of a person using a video of another person—such as footage of an actor—without any prior training on the target image.
With other face-swap technologies, like deepfakes, the algorithm is trained on the face you want to swap, usually requiring several images of the person's face you're trying to animate. This model can do it in real-time, by training the algorithm on similar categories of the target (like faces).
"I ran [the First Order Model] on my PC and was surprised by the result. What’s important, it worked fast enough to drive an avatar real-time," Aliev told Motherboard. "Developing a prototype was a matter of a couple of hours and I decided to make fun of my colleagues with whom I have a Zoom call each Monday. And that worked. As they are all engineers and researchers, the first reaction was curiosity and we soon began testing the prototype."
Aliev made a video of himself as Elon Musk, pretending to join the wrong meeting, to demonstrate the tech. It's pretty clear that it's a fake, but the eyes and head move around well enough that it'd be a neat trick for a few seconds, before the rest of the call looks any closer.
Motherboard tested Avatarify, which is available open-source on Github, and found that while the code itself is accessible, it still requires a bit of programming knowledge and some decent hardware to run the app. You have to run Zoom or Skype, as well as streaming software and Avatarify at the same time, which takes a decent amount of computing power.
"The idea after Avatarify is availability and fun. Certainly it requires a powerful gaming PC to work smoothly, but we think optimization for laptops is just a matter of time," Aliev said. "It just allows people to have some fun while being locked down at home."
To be believable, you'd need to throw your voice to impersonate the face you're making—something deepfakes still struggle with doing realistically, like that awful vocal impression of Mark Zuckerberg. But some companies, like Dessa, are nailing audio deepfakes; combined with a Zoom bombing, tech like this could seem convincingly real.