Researcher Deepfakes His Voice, Uses AI to Demand Refund From Wells Fargo

Why stay on the line with the bank for ten, 20, 30 minutes, when an AI-driven bot could waste that time for you?

That’s what a new tool from Do Not Pay promises. Do Not Pay is an organization that has previously automated all manner of things from fighting parking tickets to easily cancel unwanted subscriptions. In a video uploaded to Twitter on Wednesday, Do Not Pay founder Joshua Browder showed the tool calling Wells Fargo customer support, and using an AI-generated version of his own voice to overturn wire fees.

“We plan on making the tool publicly available with a range of generic custom voices, but to have the user’s own voice it will be a premium option,” Browder told Motherboard in an online chat.

“Hi, I’m calling to get a refund for wire transfer fees,” the fake Browder says around half way through the clip. The customer support worker then asks for the callers first and last name, which the bot dutifully provides. For a while, the bot and worker spar back and forth on which wire transfer fees the bot is calling about, before settling on the fees for the past three months.

In a tweet, Browder said the tool was built from a combination of Resemble.ai, a site that lets users create their own AI voices, GPT-J, an open source casual language model, and Do Not Pay’s own AI models for the script. Do Not Pay has previously used AI-powered bots to negotiate Comcast bills.

The conversation from this latest bot is very unnatural. There are long pauses where the bot processes what the customer support worker has said, and works on its response. You can’t help but feel bad for the Wells Fargo worker who had to sit silently while the bot slowly did its thing. But in this case, the bot was effective and did manage to secure the refunds, judging by the video.

In a statement sent after the publication of this piece, Wells Fargo said “We reviewed our call recordings, and the customer’s call to Wells Fargo didn’t happen the way it’s presented in the video. It appears the customer’s video recording of this call was edited to look like a bot was interacting with a customer service representative.” Browder then told Motherboard the video was not edited, except for removing his account number from the start of the clip.

“We called them a dozen times to test the flow,” Browder added. Motherboard asked Wells Fargo for a copy of the recording from its end, and will update if Wells Fargo provides one.

At least the bot said “thank you for your help.”

Update: This piece has been updated to include a response from Wells Fargo and more information from Browder.

Subscribe to our cybersecurity podcast, CYBER. Subscribe to our new Twitch channel.