The Bad Science Fueling a New Wave of Sports Analytics

There is no scouting myth more persistent or symbolic than "the good face." For decades, baseball scouts--who have resisted every form of technological advancement from the stopwatch to the computer--have believed that they can divine a player's future by looking at his face. It's all right there: the charisma, the aura, and the personality that will make a player a star.

But in a sports world increasingly dominated by big data and analytics, the scout's process--by their own admission more art than science--is out of place. Enter Dan Hill, the facial coding analyst the Milwaukee Bucks hired as a consultant last offseason.

As reported in the New York Times last month, the Bucks asked Hill to assist with prospect evaluation for the NBA Draft. Hill spent 10 hours with Milwaukee team psychologist Ramel Smith watching videos and cataloguing minute facial muscle movements. The team cited Hill's work as part of their rationale for selecting forward Jabari Parker, who averaged 12.3 points and 5.5 rebounds per game before tearing the ACL in his left knee last month, over guard Dante Exum.

"Nothing against Exum, but emotional resiliency, stability and an immediate, assured presence were all key considerations in support of selecting Parker," Hill told the Times.

In other words: Parker had a good face. A good face as determined by science. Hill's company, Sensory Logic, uses a "Facial Action Coding System" (FACS) developed by psychology professor Paul Ekman that examines which of the 43 face muscles are working at a given moment to determine a person's emotional state.

Featured on the television show Lie To Me and reportedly used by the FBI, police, and intelligence agencies to read subtle signs of stress and deception on the faces of potential criminals and terrorists, FACS has legitimate uses and is based on extensive, cross-cultural research of human expressions.

But does that make it a useful professional basketball scouting tool? Can the good face actually be quantified?

Dr. Kenneth Craig, a psychology professor emeritus at the University of British Columbia, has used facial action coding to successfully detect pain in extremely young medical patients.

"Our earliest attempts to use facial expression to measure pain were on young children and infants, who for the most part are unable to use words to articulate whether they're in pain or suffering or not," said Craig, who has conducted extensive facial action coding research. "It turns out facial expression is the most sensitive and the most specific source of information concerning how infants and young children are reacting. We developed very systematic ways of decoding facial activity."

Based on minute facial movements, experts like Craig not only can measure pain, but also the presence of emotional responses including fear, disgust, anger, surprise, and joy. "It requires a lot of competence on the part of the person who is doing the decoding of the expression," Craig said. "It isn't a matter of the observer or decoder interpreting what's happening--the person is upset, or moody, or that kind of thing--it's a matter of the decoder identifying the anatomic changes on the face."

So far, so good. Facial action coding is the mechanics behind the poker face, quantified. Our facial responses happen so quickly we don't have time to pass them through a filter, as we might be able to do with speech. Thus those facial responses act as tells--the sort of tells that scouts and teams have long looked for.

According to the New York Times, Hill, a Fox News contributor, has done facial analysis for 16 years, mostly for companies developing advertising campaigns or new products. After noting the sadness in Rafael Nadal's face following a crushing loss to Andy Murray in the 2008 United States Open semifinals, he reportedly had an epiphany, deciding that "he could help teams find 'the heart of a champion' by predicting and improving performance through facial coding."

Hill has since evaluated NFL Draft prospects and worked with Washington State University football coach Mike Leach. How does his system work? The New York Times reports that he examines facial expressions to identify seven key emotions--happiness, surprise, contempt, disgust, sadness, anger, and fear--and then uses his observations to create psychological profiles of players. For example:

… [Hill] categorizes smiles, for example, four ways: true, robust, weak and micro. Consider Oklahoma City Thunder guard Russell Westbrook. According to Hill, Westbrook shows a high number of weak smiles or "satisfaction," and enough "true" smiles, which equate to "joy" and contribute to what has been a highly effective season-averages of 28.6 points and 7.4 assists per game …

Hill told the New York Times that his work is particularly useful with "focus groups whose members are often unwilling to speak frankly or have trouble conveying their true opinions." And therein lies the catch. It's one thing to analyze unthinking, in-the-moment expressions to determine if a crime suspect is lying, a poker player is bluffing through a bad hand, or a NBA prospect isn't actually excited about living in Milwaukee.

It's quite another to use those same expressions to determine what kind of person someone is--particularly 18-to-22-year-old basketball players who likely are still figuring that out for themselves.

According to Craig, facial action coding is much more effective at identifying momentary feelings or states than innate or immutable characteristics. "Our moods and dispositions at any given moment in time are very much not only a manifestation of internal, stable traits, but they're also a reflection of what's happening to us at this moment in time," he said. "Where are we? Are we with friends? Are we being subjected to frightening or angering or disgusting circumstances? So you have to recognize the situational variability in expression over time."

When asked about the ability of facial coding to predict "emotional resiliency, stability, and an immediate, assured presence,"--the qualities that Hill saw as stronger in Parker than Exum--Craig was skeptical. "Those are the kinds of things that sports psychologists might say on the strength of intensive, extensive psychological assessment of the individual," he said. "That kind of assessment is likely to use personality measures, some stress testing in which there's an observation to how the individual reacts immediately to demanding situations, even those situations in which they fail, extensive interviewing of both the individual and others that know him or her well.

"I would think that there's some potential for facial expression analysis to be complementary or supplemental information derived from those other measurement approaches. But I don't think that it would be wise to arrive at conclusions of that type focusing exclusively on facial expression."

According to the New York Times article, Milwaukee team psychologist Smith conducted his own personality analyses on Parker, Exum, and other draft prospects; the Bucks merely used Hill's work to validate Smith's, and as one of many player evaluation tools. Still, it's hard not to wonder: why was the team attracted to facial coding in the first place?

From IQ exams to psychiatric questionnaires to biometrics, teams have long been willing to listen to people peddling novel, scientific-sounding talent evaluation methods--anything to identify the next great superstar, or to avoid the next great draft bust. Some of these methods have demonstrable, if limited, merit. Others reek of pseudoscientific hucksterism, like an ESPN Magazine-endorsed formula that quantifies Major League Baseball team demographics and predicts wins gained and lost due to "team chemistry scores." According to the article, San Francisco owned "the highest chemistry score (+2.0) in baseball due to well-aligned subgroups: Latino relievers of all ages, a Caucasian starting staff with different levels of experience and team tenure."

The process behind the formula is practically indecipherable. "We built a proprietary team-chemistry regression model," writes Jeff Phillips for The Parthenon Group, the company behind the formula. "Our algorithm combines three factors -- clubhouse demographics, trait isolation and stratification of performance to pay -- to discover how well MLB teams concoct positive chemistry." Demographics include age, nationality, and length of time with the team. Isolation refers to "players who are isolated because of a lack of subgroups from these shared demographic traits." The formula also includes an "ego factor" that claims to measure the impact "from individuals' differences in performance and monetary status." The conclusions imply that the correct answer is in the middle. "Teams with the highest scores have several overlapping groups based on shared traits and experiences," and, "Too much diversity can, in fact, produce clubhouse isolation for players who don't have teammates with similar backgrounds or experience."

This led to such predictions as Robinson Cano being the "unstable element" for the Mariners, who won nine more games than predicted, largely behind Cano's All-Star season, and that the isolation for Dodgers pitchers Kenley Jansen from Curacao (44 saves, 2.76 ERA) and Hyun-Jin Ryu from South Korea (3.38 ERA, 14-7 record, 139 K in 154 IP) would be an anchor for Los Angeles.

At a macro level, there may be something to the formula's claims. Isolation, for instance, can be a real issue, particularly when there isn't anybody else around who speaks the isolated player's native tongue. A Grantland story last year on the Rangers' trilingual translator Kenji Nimura, who can speak Japanese with Yu Darvish as well as Spanish and English, showed how important it can be for foreign players to have someone to talk to. But this approach flattens entire ethnic groups and races into singular entities. The assumption that Kenley Jansen and fellow Curacao natives Andruw Jones, Jurickson Profar, and Jonathan Schoop would all have the same reactions to isolation ignores the fact that every subgroup contains individuals with their own personal reactions to stressors.

When attempting to evaluate--and now quantify--the irreducibly fuzzy mental and emotional makeup of prospects, sports decision makers seem particularly vulnerable to exaggerated claims.

"The difficult piece of the puzzle is the psychological side of it," Bucks assistant general manager David Morway told the New York Times. "And not only psychological, character and personality issues, but also team chemistry issues."

On a philosophical level, reducing thinking, feeling athletes to a series of quantifiable checkmarks--racial and cultural subgroups; some facial tics in an interview--seems both demeaning and dehumanizing. On a practical level, there's no reason to conclude that it actually works. Nevertheless, sports decision makers want to believe in the good face. Same as they always have.

In the New York Times, Leach lamented that coaches were "overwhelmed with too much data that could lead to overthinking and indecision." However, he hoped that Hill's facial coding could "break through the noise."

"Someday Dan will be able to get hard data linking the face to on-the-field performance," Leach said. "And I don't want to miss that." In the meantime, the Cougars have gone 12-25 over his three seasons at WSU.