This story is over 5 years old.


How Corporations Will Use Artificial Empathy to Sell Us More Shit

Facial tics will help streamline marketing in our future consumerist dystopia.

Empathy is a tricky business. The range and complexity of human emotion makes it difficult, if not impossible, to ever really understand how someone else is feeling. Nevertheless, empathy is considered to be a crucial aspect of what makes us human—indeed, our brains appear to be hardwired for it. So perhaps it won't come as much of a surprise that as machine learning becomes ever more sophisticated and capable of mimicking some of the most complex functions of the human brain, figuring out a way to teach a computer empathy is quickly becoming a business in itself.


Known as artificial empathy, the idea here is to train machines to recognize social signals from humans, aka 'visual data,' and then produce an appropriate response. The emergence of social signal processing as a branch of computer science and robotics is a relatively new phenomenon, but it has already attracted a significant amount of attention from another field of research that is also profoundly interested in understanding the way humans communicate: marketing.

On the one hand, harnessing artificial empathy is considered an essential step toward integrating robots and artificial intelligences into human society as it will allow for more fluid and affective human-robot interaction. On the other, it is seen as a marketing goldmine, as the work being done by Shasha Lu, a lecturer in marketing at the Cambridge Judge Business School, is wont to show.

The research being done by Lu seeks to combine machine-based image and video analysis with current marketing techniques in order to streamline product sales. The idea is essentially to teach a machine to read a customer's behavioral cues in reaction to a particular product and then make targeted product recommendations based on these social signals. In Lu's case, she is specifically focusing on teaching a machine how to give clothing recommendations tailored (ha!) to the individual consumer.

"The idea of getting a machine to come to useful conclusions from someone's expressions and behavior was … massively exciting to me," Lu said in a statement. "And we now have the technology and computing power to actually make this possible."


"The idea of getting a machine to come to useful conclusions from someone's expressions and behavior was … massively exciting to me."

According to Lu, the project is based on techniques used by human salespeople.

"When people, especially females, go out shopping, we tend to try a garment on before making the final purchase," she said. "When we do so in front of the mirror, the sales person often observes two key pieces of information. The first is whether the customer likes it or not, which is inferred by her emotional response from her facial expression. The second is which particular part of the garment she likes or dislikes, which is observed through behavioral response. If she's touching her collar or scratching the lower hem with an annoying expression on her face, for example, we may infer she doesn't like that bit of the garment."

Lu's project proposes placing a camera on top of a mirror outside of a retailer's changing space, which would allow computers to capture the customer's evaluation of the piece of clothing. The customer's facial expressions and other behavioral responses would then be analyzed in conjunction with data from other customers reacting to the same clothing item in order to make other clothing recommendations to the customer in real time. Other pieces of data culled from the customer, such as height and weight, would allow the computer to make ever more customized suggestions.


Using machine learning to understand consumer behavior as a way to customize marketing campaigns is by no means a new technique, and has already been deployed to great effect in public spaces. This may take the form of face-reading billboards which use software to target shoppers (a tried and true technology at this point), or Facebook's ethically complicated algorithm which appears to racially profile users to deliver them ads tailored to their particular "affinity groups."

Facebook's tutorial on the advertising program is careful to explicitly state that the advertising scheme is not based on targeting specific ethnicities or races, but rather their "affinity to cultures they are interested in." However a quick glance at the promotional materials can make this difficult to believe, since the users pictured next to the listed affinity groups (US Hispanic, African American, Asian American) do not seem to have any shared cultural markers, although they all appear to belong to the targeted race.

A particularly salient example of this was the promotion of the film Straight Outta Compton, which used Facebook's targeted advertising to deliver two wildly different trailers to users based on their supposed affinity group (which Facebook continues to claim was not based on racial or ethnic categories).

Film trailers based on audience members' preferences or affinities is also an area that Lu is working on, which she sees as directly relevant to her targeted clothing recommendations.

"This is still ongoing, so we don't have concrete findings yet, but as it stands all movie trailers target the same group of people," said Lu. "If there's, say, a trailer with an action element that's watched by someone that doesn't go in for action films, they may well be put off seeing it, whereas in fact the movie might have plenty of non-action that makes the film attractive to this viewer. We are working on [customizing] movie trailers according to people's preferences, thereby improving the effectiveness of the movie trailer and getting people to watch movies they might otherwise dismiss."

According to Lu, similar techniques could even be extended to online dating, in order to facilitate better matching by analyzing facial characteristics. Whether these aggressively individualized marketing schemes will prove to be a bane or a boon to the consumer remains to be seen, but for her part, Lu remains optimistic: "Not only is this area of work innovative and full of potential, but it's about the everyday aspects of living, and it's this combination that makes it a really exciting area to be involved in."