Facial Recognition Failures Are Locking People Out of Unemployment Systems

People around the country are furious after being denied their unemployment benefits due to apparent problems with facial recognition technology that claims to prevent fraud.

Unemployment recipients have been complaining for months about the identity verification service ID.me, which uses a combination of biometric information and official documents to confirm that applicants are who they claim to be. The complaints reached another crescendo this week after Axios published a “deep dive” article about the threat of unemployment fraud based on statistics provided to the outlet by ID.me.

Some unemployment applicants have said that ID.me’s facial recognition models fail to properly identify them (generally speaking, facial recognition technology is notoriously less accurate for women and people of color). And after their applications were put on hold because their identity couldn’t be verified, many should-be beneficiaries have had to wait days or weeks to reach an ID.me “trusted referee” who could confirm what the technology couldn’t.

On Twitter, there are dozens of complaints about ID.me per day, and local news articles all over the country have detailed the problem over the course of months. In California, 1.4 million unemployment beneficiary accounts were abruptly suspended on New Year’s Eve and the beneficiaries were required to re-verify their identity using ID.me, a process which many found difficult and resulted in them waiting for weeks to reactivate their accounts while they struggled to make ends meet. 

In Colorado, benefit recipients who had no problem establishing their identity before ID.me took over were suddenly rejected and went months without receiving the payments they were eligible for.

The story is similar in Florida, North Carolina, Pennsylvania, Arizona, and many other states.

ID.me CEO Blake Hall told Motherboard that the company’s facial recognition technology does one-to-one matching—comparing one face against a picture of that same face (from a driver’s license, say)—whereas other applications of facial recognition attempt to find a match for a face in a large dataset of faces, known as one-to-many matching.

“The algorithms used for Face Match operate ~99.9% efficacy,” Hall wrote in an email to Motherboard. “There is in fact no relationship between skin tone and Face Match failure on a 1:1 basis” according to a regression analysis the company performed.

That doesn’t mesh with the experiences being shared on Twitter by people like Tim Weaver, a gig economy worker in Las Vegas who was suddenly cut off from his unemployment benefits in late March after ID.me failed to identify him.

Weaver told Motherboard that when he attempted to pass ID.me’s facial recognition test he held a phone in front of him in the instructed position but “it rejected it, didn’t give us a reason, just rejected it. It rejected it three times, and then it locked me out of the system.”

Weaver said he attempted to contact the company’s customer support through its chat feature, which claims to provide assistance 24-hours a day, seven days a week. He tried numerous times at all hours of the day. He tried contacting the state of Nevada for help, but the employees there directed him back to ID.me.

This went on for several weeks, Weaver said, until he tweeted a scathing criticism of the company, which then reached out and—after several more frustrating days—verified Weaver’s identity.

Weaver went for three weeks without receiving his benefit. “I couldn’t pay bills,” he said. “Luckily I had enough food saved up so I didn’t have to worry about that. It’s just ridiculous.”

In his statement to Motherboard, Hall said that facial recognition failures are not a problem with the technology but with the people using it to verify their identity. “For example, if someone uploads a selfie that only shows half their face.”

“We are unaware of eligible individuals who have not been able to verify their identity with ID.me,” he said, and “the wait time for a live video chat session right now is less than five minutes and it has been consistently under 30 minutes all week.” 

A quick search for tweets directed at ID.me shows reams of people locked out of their unemployment accounts by the company.

At least 21 states use ID.me to detect fraudulent benefit claims. The spread of the technology has coincided with an aggressive media blitz from Hall in which he has stoked fears about the crisis of unemployment fraud using some fluid statistics.

In February, he told an Oregon TV station that unemployment fraud had cost the country $100 billion. Several weeks later, he told a Montana TV station that unemployment fraud had cost the country nearly $200 billion. The next month, he was telling a San Diego station that unemployment fraud would cost $300 billion. By the time ID.me reached Axios, it was $400 billion.

Hall told Motherboard that the Oregon TV station didn’t quote him directly—it did attribute the $100 billion estimate to him, though—and that he was consistent through January and February estimating the losses from unemployment fraud around $200 billion. “As we screened existing claimants in more states, we had more data points … In June, I estimated the loss over $400 [billion] in line with past estimates. All of those data points are consistent over time.”

Hall did not respond to Motherboard’s question about how ID.me calculated its estimates.

It’s possible that the rate of successful unemployment fraud is growing rapidly—hence Hall’s rising cost estimates—but ID.me also has a vested interest in depicting fraud as a major problem, which Hall has repeatedly done. It makes sense for a company that makes anti-fraud technology to study fraud; it's also fair for the general public to be skeptical of fraud statistics published by companies that would benefit from showing that fraud is a major problem. 

Axios has now published two articles that have cited the company's numbers, and, on Twitter, Axios writer Felix Salmon suggested that fraud associated with expanded unemployment insurance during the pandemic has resulted in "quite possibly the largest theft of all time." 

The Axios article has been widely criticized for using ID.me—a company with a financial incentive to inflate the rate of unemployment fraud—as its main source. And while many local news outlets have provided great coverage of the pain caused by delays due to ID.me’s system, others have run with stories similar to Axios’s, citing the company’s numbers. The Axios article was also immediately cited by Republicans in the House of Representatives as evidence of widespread defrauding of the U.S. government. 

The U.S. Department of Labor says that between March and October 2020 it uncovered just $5.6 billion in potentially fraudulent unemployment payments. And while more recent data suggests the losses are actually higher, the agency estimated that they would be “into the tens of billions of dollars.”

Weaver said he thinks it’s very important to prevent fraudulent claims, but he’s mad at ID.me and even madder at Nevada for wthe current identity verification system and how it was implemented on such short notice. 

“They knew there were going to be a million problems,” he said.