Proctorio Is Using Racist Algorithms to Detect Faces

A student researcher has reverse-engineered the controversial exam software—and discovered a tool infamous for failing to recognize non-white faces.
April 8, 2021, 4:49pm
A girl wearing a yellow shirt sits at a brown wooden desk and writes on paper while attending a virtual class on her laptop
John Moore | Getty Images

Students of color have long complained that the facial detection algorithms Proctorio and other exam surveillance companies use fail to recognize their faces, making it difficult if not impossible to take high-stakes tests.

Now, a software researcher, who also happens to be a college student at a school that uses Proctorio, says he can prove the Proctorio software is using a facial detection model that fails to recognize Black faces more than 50 percent of the time.

Advertisement

“I decided to look into it because [Proctorio has] claimed to have heard of ‘fewer than five’ instances where there were issues with face recognition due to race,” Akash Satheesan, the researcher, told Motherboard. “I knew that from anecdotes to be unlikely … so I set out to find some more conclusive proof and I think I’m fairly certain I did.” 

Satheesan recently published his findings in a series of blog posts. In them, he describes how he analyzed the code behind Proctorio’s extension for the Chrome web browser and found that the file names associated with the tool’s facial detection function were identical to those published by OpenCV, an open-source computer vision software library.

Satheesan demonstrated for Motherboard that the facial detection algorithms embedded in Proctorio’s tool performed identically to the OpenCV models when tested on the same set of faces. Motherboard also consulted a security researcher who validated Satheesan’s findings and was able to recreate his analysis. 

On its website, Proctorio claims that it uses “proprietary facial detection” technology. It also says that it uses OpenCV products, although not which products or what for. When Motherboard asked the company whether it uses OpenCV’s models for facial recognition, Meredith Shadle, a spokesperson for Proctorio, did not answer directly. Instead, she sent a link to Proctorio’s licenses page, which includes a license for OpenCV.

Advertisement

“While the public reports don’t accurately capture how our technology works in full, we appreciate that the analyses confirm that Proctorio uses face detection (rather than facial recognition),” Shadle wrote to Motherboard in an email.

Proctorio also did not answer several questions about the technology, including whether or not the company had fine-tuned the OpenCV models for use in its software.

The failure of computer vision systems to detect and correctly identify darker-skinned faces is a well-documented issue, and Proctorio’s use of OpenCV’s models has severe ramifications for the students forced to use it. Facial recognition and detection software that have been built off of OpenCV have previously been found to be biased.

Satheesan tested the models against images containing nearly 11,000 faces from the FairFaces dataset, a library of images curated to contain labeled images representative of multiple ethnicities and races. 

The models failed to detect faces in images labeled as including Black faces 57 percent of the time. Some of the failures were glaring: the algorithms detected a white face, but not a Black face posed in a near-identical position, in the same image.

Two women standing next to each other smiling. The woman on the left, who is white has a blue box imposed around her face, while the Black woman on the right does not.

The pass rates for other groups were better, but still far from state-of-the-art. The models Satheesan tested failed to detect faces in 41 percent of images containing Middle Eastern faces, 40 percent of those containing white faces, 37 percent containing East Asian faces, 35 percent containing Southeast Asian or Indian faces, and 33 percent containing Latinx faces.

Black students have described how frustrating and anxiety-inducing Proctorioi’s poor facial detection system is. Some say the software fails to recognize them every time they take a test. Others fear that their tests will abruptly close and lock them out if they move out of perfect lighting.

“There’s no reason I should have to collect all the light God has to offer, just for Proctorio to pretend my face is still undetectable,” one student wrote on Twitter.

Satheesan said his findings help demonstrate the fundamental flaw of proctoring software: that the tools, ostensibly built for educational purposes, actually undermine education—particularly for already disadvantaged students.

“They use biased algorithms, they add stress to a stressful process … during a stressful time, they dehumanize students,” he said.