Every face is different. The exact space between eyes, the precise curve of the cheek, the fullness of the lips — the sum of all these features distinguishes one person from another, even identical twins.
With a decent camera and a simple mathematical algorithm, any person's face can be analyzed and converted into an individualized "faceprint" — a unique identifying tag much like a fingerprint.
Capturing, storing, and ultimately selling facial biometrics has quickly become big business. A recent research report valued the global facial recognition market at $1.3 billion in 2014. It could double by 2022.
For now, this market is completely unregulated by the United States government.
"Here's what's scary: facial recognition tracks you in the real world — from cameras on street corners and in shopping centers, and through photographs taken by friends and strangers alike," Minnesota Sen. Al Franken told VICE News. Franken is the ranking Democrat on the Senate Subcommittee on Privacy, Technology, and the Law. He's been keeping an eye on facial recognition technology, and thinks it's time for stricter oversight.
"What we really need are federal standards in place that directly address facial recognition privacy," he said.
No specific legislation has been proposed, however. Apart from concerns voiced by internet privacy advocates, consumer organizations have been largely silent on the issue.
"There's a real lack of appreciation for the impact of this technology on consumers," Dr. Joseph Atick, a pioneer of facial recognition, told VICE News. A scientific prodigy who enrolled in a Ph.D. program at Stanford at the age of 17, Atick helped develop the first facial recognition system in the world and founded Visionics, one of the first companies in the field, in 1994. Today he hopes to reign in the industry he helped create.
'Internet companies that get their revenue from advertisements and not from consumer purchases don't have a lot of natural incentives to safeguard their users' data.'
Advertisers already track your every move on the internet — from your search terms to the subject lines of emails. Companies like Facebook, Shutterfly, and LinkedIn are also collecting millions of images that are run through facial recognition algorithms and tagged with a unique code.
"What I'm worried about is that they will link my offline and online profile, targeting me as I walk down the street in ways that are invasive. I call that a 'republic of fear'," Atick remarked. "I did not invent this technology 30 years ago with that in mind, that's for sure."
The technology has improved significantly since Atick founded Visionics, which merged with another biometrics identification company in 2001. Google's FaceNet algorithm reportedly identifies faces with 99.63 percent accuracy. Facebook's DeepFace works 97.25 percent of the time. Both systems significantly outperform the FBI's own recognition program, which reports an 85 percent success rate.
The federal government is playing catch-up with the private sector in more ways than one. In July, the Government Accountability Office released its first report on commercial facial recognition technology, which admitted that it wasn't sure how common the technology was.
"Facial recognition technology is currently being used in a number of commercial applications in the United States," the report read, "but the full extent of its present use is not known."
Some states have moved to regulate the use of this technology. Illinois and Texas both require that companies ask permission before collecting or using a person's biometric information, and two lawsuits are pending in Illinois against both Facebook and Shutterfly for collecting users' facial data without informed consent. A principal concern is that the companies are more or less writing privacy rules — or not — as they go along.
"The legal framework is very wide open," Brian Brackeen, the CEO of the facial recognition company Kairos, told VICE News. "It's the wild west."
Kairos uses the technology to scan shopping malls to see how long people linger in front of window displays and analyze the gender breakdown of crowds. The tools can measure attention and detect emotions and ages.
'We recognize the creepy, but we don't want to stifle innovation. If we cross that line from cool to creepy, people will stop using that service.'
As the technology continues to advance, the debate over its future is becoming increasingly polarized. Internet privacy watchdogs like the American Civil Liberties Union and the Electronic Frontier Foundation warn of a rapidly expanding and largely unregulated commercial surveillance apparatus. Tech industry lobbyists, meanwhile, herald facial recognition as a natural next step in streamlining the consumer experience.
The two sides have been meeting periodically since 2014 as part of a dialogue convened by the National Telecommunications and Information Administration. The meetings — dubbed a "privacy multistakeholder process" — are designed to produce voluntary guidelines for the facial recognition industry as a whole. But the talks broke down in mid-June, when eight different privacy groups walked out to protest what they called industry intransigence.
Alvaro Bedoya, the director of the Center for Science and Privacy at Georgetown Law School, led the walkout. He told VICE News that the disagreement boiled down to the question of consent.
"There's no requirement for companies to get your permission before they use facial recognition software on you," he said, adding that companies seem to be ignoring state-level restrictions.
At the June meeting, he asked industry representatives if they would be willing to create a system that would help companies ask permission from users before photos of them would be used to make a faceprint.
"I was met with silence," he said.
Carl Szabo, a lawyer with NetChoice — a tech industry group that represents companies like Facebook, Google, and Yahoo — was also in attendance. He thinks consent might well be a non-starter for consumers. From the industry's perspective, requiring permission at every step of the process — from the gathering of facial images online to the use of camera systems — would prove too cumbersome.
"We don't know if consumers want a pop-up notice every time they upload a photograph to a service, or if consumers want to sign a form every time they enter a store," he told VICE News.
Szabo also suspects that privacy advocates are gunning for an outright ban on facial recognition technology. "Some people may want to make this technology illegal before it has a chance to grow," he said. Though he acknowledged that balancing privacy and progress is a challenge, he expressed confidence that the tech industry is committed to giving consumers "meaningful control" over their faceprints.
"We recognize the creepy, but we don't want to stifle innovation," he remarked. "If we cross that line from cool to creepy, people will stop using that service."
Szabo and the companies he represents predict that privacy concerns will fade as consumers begin to understand the benefits of the new technology. "Imagine you walk into a Nordstroms, a camera scans your face, recognizes you, and knows what shirt you bought on your last visit," he said. "Then, the salesperson can recommend a matching tie — that's just good customer service."
After the talks stalled in June, Franken penned a letter critical of the industry's position. "I hoped that private sector participants would recognize the importance of Americans' basic right to decide who may collect and utilize their sensitive biometric information for commercial purposes," he wrote. He then suggested that the logjam might reflect "the potential need for federal legislation to adequately protect the privacy of consumers' biometric information."
While the multistakeholder process remains in limbo, companies are increasingly employing facial recognition for targeted advertising and to generate sales. Microsoft declared 2015 the year of "on-demand" and "personalized" shopping, announcing plans to deploy facial recognition technology in retail outlets to help stores make stocking decisions. It has already patented technology for billboards that can scan passersby and tailor ads accordingly. Hotels in the UK use facial recognition to remind concierges of guest names, and casinos are using the software to tag and identify cheaters.
But for now, the scenario described by Atick in which facial tracking links a consumer's online and offline profile is largely hypothetical.
Offline facial recognition software is used to identify very specific sets of people: registered hotel guests, employees at a company, even churchgoers. Companies like Kairos that scan public places tend to only report macro-data, such as the gender breakdown of a crowd. This is partly because they don't have access to a massive database of faces. Facial recognition data is much more organized online. Millions of photos are posted on the internet or social media and archived by companies like Facebook and Google, tagged wherever possible with names and personal information.
'Every person has a unique pattern that they were born with. That pattern must be your personal property.'
There are no laws in place to keep the offline and online words separate. But the sharing of data across these channels could make walking down the street more and more like browsing the internet, where each person's movements are trackable, searchable, and for sale to advertisers.
"Internet companies that get their revenue from advertisements and not from consumer purchases don't have a lot of natural incentives to safeguard their users' data," Bedoya said. "Their interests just don't align with the consumer."
He thinks that all of the elements are in place to merge the two into a juggernaut surveillance system that would amount to a wet dream for advertisers.
"It's a pot of gold that's so, so tempting," Atick agreed. "I'll soon be walking down the street and my phone will be constantly buzzing with ads saying, 'By the way, your girlfriend's birthday is coming up. You just looked in a storefront — why not go buy something?' "
For that to happen, a massive database of faceprints need only be on the market for advertisers to snap up or connected to offline recognition systems — and Facebook happens to own such a collection.
The company began collecting its users' facial biometrics in 2010. Though it will not reveal the extent of its stockpile, the company has acknowledged that it stores more than 250 billion user-uploaded images. One of its engineering directors has called it "the biggest human dataset in the world."
Technically, Facebook users can opt-out of being subject to facial recognition, but the option is buried in the site's privacy settings, and the words "facial recognition" don't appear anywhere. To disable facial recognition, users must navigate their way to a sub-menu titled, "Who sees tag suggestions when photos that look like you are uploaded?"
VICE News asked Facebook if it expects to sell faceprints to advertisers or make its trove available to other facial recognition systems.
"We have no plans to use these templates outside of the Facebook ecosystem," a company spokesperson wrote in reply. "We're strongly committed to giving people control over their information, which is why we make it easy for people to turn off tag suggestions (and delete the associated template) in their settings."
This feature is not available in Europe — the company withdrew the service in 2012, after regulators asked it to provide a more transparent "opt-in" mechanism.
Facebook only uses tagging software for internal use, but it has not ruled out monetizing its massive database. In 2013, Al Franken asked Rob Sherman, Facebook's manager of privacy and public policy, if the company planned to sell facial data to third parties.
"It's difficult to know in the future what Facebook will look like five or 10 years down the road, and so it is difficult to respond to that hypothetical," he replied. "Can I say that we will never use facial recognition technology for any other purposes? Absolutely not."
Facebook would have to change its privacy policies to sell this data, which could prove challenging — at least in the near term. Since 2011, the company has been under a federal consent decree after the Federal Trade Commission found that it had engaged in "unfair and deceptive" practices by publicizing information users had posted on their Facebook pages and deemed private. The company's privacy policies must be independently vetted until 2021.
"It will be very difficult for anyone to recreate Facebook's level of database anytime soon," Bedoya said. But he's still convinced that legislation requiring consent is necessary to safeguard privacy. "We need some basic privacy laws in place, and then we can develop some voluntary best practices."
Szabo, the tech industry lawyer, disagrees.
"Legislation cannot move at the speed of innovation," he remarked. Instead, he suggests companies make their facial recognition policies hyper transparent so that consumers can "vote with their feet" if they are "creeped out."
While Atick doesn't favor immediate legislation, he does want companies to completely revolutionize the way they treat photographs.
"Every person has a unique pattern that they were born with. That pattern must be your personal property," he said. "If we don't start thinking that way, pretty soon I'll start wearing a hat and sunglasses when I walk out the door, because there will be no place to hide."
Follow Avi Asher-Schapiro on Twitter: @AASchapiro