This Tool Will Tell You If You Can Make a Living Wage on Mechanical Turk

Essential workers have been in the spotlight throughout the COVID-19 pandemic, receiving politicians’ praise and fighter jet fly-bys while struggling with low pay and dangerous working conditions.

Now, one researcher has built a tool to highlight another particular group of underpaid workers: micro-taskers who spend their days sorting and labeling data for services like Amazon Mechanical Turk.

The tool—called the Technically Responsible Knowledge wage calculator, or TRK for short—allows anyone to check if the various data entry and labeling tasks being offered on services like Mechanical Turk result in a living wage. Perhaps unsurprisingly, many do not.

“While companies like Amazon and Facebook do rely on highly-paid computer scientists, they also rely on a sprawling workforce that internet users don’t hear about,” wrote Caroline Sinders, an artist and researcher who created the project with a small team, supported by Mozilla. (Disclosure: Sinders has written for Motherboard in the past). “Around the world, many thousands of gig workers label data and train algorithms that power consumer technology. And these gig workers are rarely paid a fair wage or provided with basic benefits.”

Sinders explains that she was inspired to highlight the human labor behind machine learning projects after having conversations with fellow designers and micro-tasking gig workers on platforms like Mechanical Turk and Fiverr. Data scientists often utilize these services as a human labor resource, using human operators to label data in order to train their machine learning systems, and setting the amount paid for each completed task—like describing the contents of an image, for example.

Then, Sinders started doing micro-task gig work herself.

“That’s when I realized that even people who thought they were pricing fairly probably weren’t, because the concept of time wasn’t included in the calculation,” Sinders told Motherboard. “So many research labs use Mechanical Turk. I kept coming back to this thought of, when people make responsible technology, are they aware of how irresponsible it can be?”

It turns out that labeling data isn’t just a mindless task. While researching for the project, Sinders completed about 40 hours of gig work on Mechanical Turk and timed how long it took her to complete tasks. She then factored in the amount of time spent on each task, so the TRK calculator will notify a user when the assigned task is paying below minimum wage. Sinders set the rate at $11 per hour, which is the minimum wage for the state of Washington, where Amazon is based. The tool features three sliders, allowing users to adjust the time and pay per task, as well as the number of tasks to be completed.

Pay disparity for gig workers is especially concerning now, with tens of millions of people unemployed due to the COVID-19 pandemic and many others working app-based gig jobs that don’t pay a living wage. According to a New York University study, 70 percent of Amazon Mechanical Turkers are women, and 65 percent live in households with an annual income under $60,000, compared to 45 percent of the general population.

Sinders says TRK is still in its alpha stages, and doesn’t take into consideration several factors, like the higher taxes paid by freelance workers in many countries. But she hopes the tool will be a provocation that causes researchers and companies to rethink their concepts of fairness when it comes to the gig workers providing these services.

“I want this to be useful for Turkers, but I also wanted to provide something for clients who wanted to try to be better but just lacked the available tools,” Sinders told Motherboard. “This is a way, I think, to get closer to one slice of equity within technology.”