Jean-Philip Poulin was feeling "joyful" and "excited" when I interviewed him recently in Montreal. I know this because he showed me his real-time emotion metrics during our conversation, which were being parsed by a machine-learning algorithm that uses heart-rate data transmitted from his Microsoft Band 2 fitness tracker. "Hey, you're feeling good!" I exclaimed, and his "joyful" levels spiked a little.
Poulin is the COO of Sensaura, a Montreal-based software startup that proposes to bridge the gap between consumer wearables and affective computing. If its founders are as successful as they believe they will be, their product will hasten the inevitable future of emotionally intelligent machines: video games will know when you're bored, advertisers will know when you're swayed, and mental health professionals will know when you need a check-in.
So far, progress in affective computing has depended on facial recognition software, which reads people's emotions pretty much the same way that people do: by looking at their faces for cues. While many researchers and companies are already using those programs, Poulin suggests that the technique is limited for two reasons. First, people can control their facial expressions and therefore dupe machines, just as we fake or conceal our emotions from each other. And second, people who are concentrating on something—an opponent in an RPG, for example—might have muted expressions, despite all the inner thrills and chills they're experiencing.
Sensaura and software like it could be the holy grail for advertisers
You can fake a smile, but not a heartbeat. Unfortunately for the manufacturers of activity-monitoring wristbands, heart rate has proven to be a pretty useless metric, at least when it comes to delivering the profound self-awareness that's supposed to save us from our bad behaviours.
Heart-rate variability is much more revelatory. These tiny differences in the intervals between heartbeats have been used as a measure for stress levels since the 60s. But this measurement is also a lot more subtle. You can't get it from your wrist, which is why some companies are trying to find a way to make it easy to attach sensors to our chests.
The founders of Sensaura believe they've navigated these problems by forcing crappy wrist-based data to give the real emotional goods. (So far, it works with Adidas miCoach, the Basis fitness tracker, and some other models, but not yet with Fitbit.) The key is a proprietary algorithm developed by the company's CTO Mojtaba Khomami, a PhD candidate at the University of Trento in Italy, who studies affective computing and neuroscience.
The Sensaura algorithm runs on what Khomami calls a "machine-learning engine," or in other words, artificial intelligence. With this, the Sensaura app performs pattern analysis on data from a tracking device. The founders claim the algorithm can detect joy in a person's heart-rate as easily and accurately as we can see it in a smile on her face.
Sensaura already performs consulting services for game developers and marketing agencies who want to understand how users are connecting with their products, but in the big picture, they are pursuing partnerships with device manufacturers to add better functionality to existing services. You may never download the Sensaura app, but if your name-brand wearable starts telling you how you feel, it may be using the company's software.
"One day, we want to be the engine behind all the connected devices that we can sense from," Khomami told me.
It's easy to spot the cynical uses of affective computing, especially once it becomes readily available through consumer fitness monitors. Tracking devices won't be faddish forever; as soon as wearables start offering better tools than step-counting, such as measuring stress levels, and once the data they provide becomes interesting to other companies that are willing to incentivize them—in other words, to distribute them for free, adding points and discounts for their products—we'll see more adoption and stronger traction.
Sensaura and software like it could be the holy grail for advertisers, who will be able to adapt their interactions based on the user's emotional response. If marketing isn't a science yet, it will be.
Rafael Calvo thinks affective computing can do better than to make us vulnerable to companies that want to prey on our emotions. Calvo, a professor at the University of Sydney in Australia, has a PhD in artificial intelligence, and focuses on creating software systems "that support rather than hinder psychological well-being and mental health," he told me over Skype.
He's part of a Positive Computing movement, whose premise is that design for well-being should be part of the design for all technology in the future. Calvo's guiding principle is that affective computing should promote human autonomy, such as tutoring software that knows how the student feels and adjusts accordingly to create appropriate challenges, or programs that make it easier for workers in "male-dominated industries" to overcome the stigma of seeking help for mental health. In this vision, kinder, gentler computers will offer customized assistance on our path to becoming more competent and confident people.
The dream of positive computing may seem like a fantasy, but consumers prefer technologies that give them a feeling of control. The alternative won't make anyone joyful or excited. "[Agency] is a huge factor in psychological well-being," Calvo said. "If you feel manipulated, you're much more likely to experience mental illness."