The User Experience Researcher Who Was Fooled by a Twitter Bot

In light of Microsoft's recent ill-fated attempt at creating a teen girl chatbot, I found myself wondering how often these programs pass themselves off as real people. I reached out to a user experience and interaction researcher, who asked to remain anonymous because of her employer's policies, to find out if she could tell me any stories about people who had been fooled by bots. This researcher struck me as more tech-savvy than your average person, so I was surprised when she told me she had briefly thought the chatbot Olivia Taters, whose Twitter feed reads like the unedited thoughts of the British teenager inside all of us, was a real person. We spoke over Gchat.

MOTHERBOARD: What was it like when you started talking to @oliviataters?
So I've never actually spoken to Olivia Taters. I just saw a bunch of retweets of hers from other people. I think I've maybe actually tweeted at her once or twice, but for the most part, I had seen tweets of hers, etc.

What convinced you there was a real person behind those tweets?

It was when I didn't actually know much about bots and had only briefly played around with Markov chains, so I knew that there were pretty good bots off Twitter that could engage in things close to human conversation (such as semi-nuanced answers and questions) but I didn't realize that could live on Twitter. Markov chains sound really awkward most times, but Olivia's tweets that people retweeted were really really funny and sounded very "real." It didn't read like a Markov chain. So I just thought this was some weird person in the internet.

Yeah, I've definitely seen real people's Twitter accounts that make less sense than Olivia. For those who don't know, could you explain what Markov chains are?

There's a really formal definition of a Markov chain that involves knowledge chains and memorylessness, but in short, it's predicting words from the last word.

The Markov chains I was using were from my past tweets, like an ebooks. Tl;dr it was my random old tweets smushed together based off programming to make sentences. So those were often really weird sounding and off, but Olivia's were pretty nuanced! So I just assumed [she was an] internet weirdo until a person I was dating at the time told me Olivia was a bot.

I also thought boodooperson on Twitter was a bot, but that was only really briefly.

How long did you think Olivia was real? And how did you feel when you found out she wasn't?

I thought Olivia was "real" for like a month maybe? I wasn't really sad. I laughed at myself and was like, "her creator is awesome," and I got really into bots after that. I really like the blurred line between a bot and a human.

In retrospect, what signs do you think you overlooked?

Probably checking out her Twitter profile! I should have done that.

Haha, yeah that might have given you a good clue. Going back to Markov chains, how can these be employed to make bots that are less awkward and more nuanced, like Olivia?

Well it's not really Markov chains but more like better programming of them? And all other kinds of machine learning algorithms to use. There's a lot of ways to make bots. Around the time I discovered or contacted Olivia, I was only aware of like Markov chains and ebooks and bots like everyword.

So Olivia was more sophisticated than bots you'd encountered before?

Err maybe? I would say whoever wrote the things it can say is really sophisticated, and that could be a combination of programming and the actual words it can write.

As someone who deals with user interfaces, what lessons did you take away from this experience?

Well not a lot because it's on Twitter. But I was under the impression Olivia was an art project and not just an experiment.

And I also never tweeted at Olivia…

How do you think interacting with Olivia would have changed your impressions?

I don't know if it was possible to, or if it was programmed to respond. But like, plenty of people tweet at accounts that don't get responses, so that would not have signaled that that account was a person or not. And to caveat I had only seen like a handful of Olivia's tweets via retweets so this was like an entity peripherally in my sphere.

In a recent interview about the Tay debacle, Olivia's creator, Rob Dubbin, said he tries "to do things that are positive in nature and joyful and celebratory and break Twitter in ways that make people laugh and make people happy." In light of this, how do you feel about the time you thought Olivia was a human?

Very pro that! Haha Olivia was and is really funny, and it was very enjoyable to read its tweets. I mean, it made me laugh and not because of broken programming or awkward phrases, but from things that were genuinely funny. Olivia Taters has probably "written" funnier tweets than me.

Yeah, I agree that she can be really funny. Do you think humor is part of what makes her more believable?

Could be!

Do you tend more towards skepticism or wanting to believe?

Um, neither? I mean, I work with bots now as a day job, so I'm just endlessly curious. I play around with retail chat interfaces to legitimately see what feels real and not because I enjoy it, and I ask those people/entities if they are real or not.

In your experience, what qualities make bots more convincing?

The ability to give more specific answers based off the questions I ask. Like, "How does it fit? Is it soft or true to size?"

When people interact with these systems, do they tend to start off believing they're real? Or assuming they're bots?

I can't really say. I assume most people assume it's a bot now, especially in e-commerce.

Yeah that makes sense. I guess what I'm wondering is whether people want it to be real.

Sometimes, I think. Especially in e-commerce settings.

Update: This story has been updated with additional quotes to reflect how little interaction the researcher had with @OliviaTaters, and to remove the researcher's name.

For April Fools' Day, we're doing stories and interviews about trickery. Check 'em out here.