Citizen logo in the sky
Image: Jason Hoffman
Tech

'FIND THIS FUCK:' Inside Citizen’s Dangerous Effort to Cash In On Vigilantism

Internal documents, messages, and roadmaps show how crime app Citizen is pushing the boundary of what a private, app-enabled vigilante force may be capable of.
Screen Shot 2021-02-24 at 3
Hacking. Disinformation. Surveillance. CYBER is Motherboard's podcast and reporting on the dark underbelly of the internet.

Andrew Frame was excited. 

It was Saturday night two weeks ago, and Frame, the CEO of the crime and neighborhood watch app Citizen, was on Slack, whipping himself and his employees into what he'd later call at an all-hands meeting a "fury of passion" about a wildfire that had broken out earlier that afternoon in Los Angeles' Pacific Palisades neighborhood. 

Citizen had gotten a tip that the wildfire was started by an arsonist, and Frame had decided earlier in the night that the fire was a huge opportunity. Citizen, using a new livestreaming service it had just launched called OnAir, would catch the suspect live on air, with thousands of people watching. Frame decided the Citizen user who provided information that led to the suspect’s arrest would get $10,000. Frame wanted him. Before midnight. As the night wore on, Citizen got more information about the supposed suspect. They obtained a photo of the man, which they kept up on the livestream for large portions of the night. More information trickled in through a tips line Citizen had set up. (Citizen said "The information about the person of interest came from an on-the-ground tip from an LAPD Sergeant, followed by emails from local residents who had been approached by LAPD officers.")

Advertisement

"first name? What is it?! publish ALL info," Frame told employees working in a Citizen Slack room who were working on the case. 

"FIND THIS FUCK," he told them. "LETS GET THIS GUY BEFORE MIDNIGHT HES GOING DOWN."

"BREAKING NEWS. this guy is the devil. get him," Frame said. "by midnight!@#! we hate this guy. GET HIM."

He was growing impatient. He increased the bounty to $20,000. Thousands of people were watching Citizen's livestream, but the man still hadn't been caught. Frame asked his staff to send out another notification, one that would hit all Citizen users in Los Angeles. The bounty had to go higher.

"Close in on him. 30k Let's get him. No escape. Let's increase. 30k," Frame said. "Notify all of la. Blast to all of la."

"Citizen is OnAir: Arsonist Pursuit Continues," the notification, which went out to 848,816 Citizen users in Los Angeles, said. "We are now offering a $30,000 reward for any information directly leading to his arrest tonight. Tap to join the live search."

Over the course of nearly seven hours, Citizen, under the increasingly frantic direction of Frame, conducted a citywide, app-fueled manhunt for a specific suspected arsonist. The employees went back and forth on how they should frame the manhunt they had started, who in Los Angeles they should notify via the app, and how often they should do it. 

Do you work at Citizen? Do you have access to internal Citizen documents? We’d love to hear from you. Using a non-work phone or computer, you can contact Joseph Cox securely on Signal on +44 20 8133 5190, Wickr on josephcox, OTR chat on jfcox@jabber.ccc.de, or email joseph.cox@vice.com.

Advertisement

In the Slack room with Frame, one staffer brought up a "loophole," pointing out that Citizen was violating its own terms of service that prohibit "posting of specific information that could identify parties involved in an incident." The staffer who brought up the terms of service violation was ignored in that specific Slack room, and the broadcast continued to specifically name the person and share his photo for hours.

Earlier in the night, soon after news of a fire broke, Frame said he saw the fire as a chance to catch a suspected arsonist live on the internet, therefore proving Citizen's utility to users and helping the app grow. 

"The more courage we have, the more signups we will have. go after bad guys, signups will skyrocket. period ... we should catch a new bad guy EVERY DAY," Frame said. 

At one point, Frame said "these metrics will be great." And they were. At one point 40,000 people were watching the live feed, according to the Slack messages. Citizen saw a sharp spike in signups as the livestream spread. Frame said at a later all-hands meeting that 1.4 million people engaged with the content, according to other Slack messages.

Well after midnight, Los Angeles police made an arrest. In a separate Slack room, employees cautiously began to celebrate: "cop said its an ongoing investigation, this looks like our guy!!!" one employee wrote.

It wasn't Citizen's guy. Frame and the entirety of the Citizen apparatus had spent a whole night putting a bounty on the head of an innocent man. 

Advertisement

(Motherboard is not publishing the name of the person Citizen falsely accused, though Citizen repeatedly used it both internally and externally.)

Motherboard spoke to eight sources in reporting this story: five former Citizen employees, two sources with knowledge of the company's operations, and one person close to the company's founders. Motherboard also obtained multiple caches of internal policy documents, Slack messages, and company notes. Our reporting spells out not only what happened in Pacific Palisades, but also how workers and Andrew Frame view the incident and Citizen's role in society. The app pitches itself as a public-safety tool, but aims to grow its user base and revenue just as much as any other startup. The Palisades incident was characterized by Frame as a risk, a test, an experiment, even though it potentially put the person they named in danger.

Motherboard has learned that: 

  • Users are flooded with notifications in what multiple sources interpret as an attempt to make users feel anxious enough about their neighborhoods to buy "Protect," a $19.99 per month service that allows users to livestream their phone's camera and location to a Citizen "Protect agent" who monitors it and sends "Instant emergency response" in case of an emergency.
  • The return of a missing autistic teen to his family in the Bronx earlier this month was done by Citizen employees on a "Street Team" that films and interacts with people while pretending to be ordinary app users.
  • Employee performance is measured by how many seconds it takes workers to input an incident into the app and how many incidents they cover.

Advertisement

Citizen's grand vision has never been a secret: From its initial launch as an app called "Vigilante" in 2016, the company pictured a world in which people were alerted to crime as it happened, and then app users stepped in to stop it before the police needed to intervene. In the Vigilante launch advertisement, a criminal stalks and then attacks a woman in New York City. The app broadcasts the location of this active crime to Vigilante's users, and a horde of people descend on the criminal, stopping the crime in progress: "Can injustice survive transparency?" the ad asks.

Thus far, however, Citizen has essentially been a social network for reporting crime that operates in around 50 cities. Citizen workers listen to and summarize police scanner audio as "incidents," which are then pushed to the app. Users can also post their own incidents, upload photos and videos, and comment on or react to incidents with emojis. The app allows users to search "around you" for incidents, and also sends push alerts to users for nearby events. 

"The whole idea behind Protect is that you could convince people to pay for the product once you’ve gotten them to the highest point of anxiety you can possibly get them to," one former employee said, referring to Citizen's subscription service. "Citizen can’t make money unless it makes its users believe there are constant, urgent threats around them at all times," they added. A Citizen spokesperson denied this in a statement: "It’s actually the opposite. With user feedback in mind, we have designed the Citizen home screen so users only see relevant, real-time information within their immediate surroundings," the spokesperson said.

Advertisement

The disastrous Palisades fire bounty hunt and the discovery of a Citizen-branded "private patrol" vehicle driving around Los Angeles (part of a pilot program in which Citizen envisions offering a physical private security force to respond to the problems of its users) hint that Citizen's goals essentially remain the same as Vigilante's. 

Frame seems to imagine Citizen as an all-encompassing crime-fighting machine that he believes will make the world safer. In Slack messages viewed by Motherboard, Frame calls ProtectOS, the system Citizen uses to create incidents and push them out to users, "the most powerful operating system ever created." 

"Our vision is a global safety network of people protecting each other—a world in which a kidnapping is impossible, because everyone is looking out for each other, and neighbors are alerted as soon as a kidnapping attempt is made," a Citizen spokesperson told Motherboard in a statement.

Vigilante was instantly controversial for a variety of reasons. Unsurprisingly, police said the app encouraged vigilantism. Critics worried that the app's users would racially profile Black people as suspicious, as happened on other safety-focused apps. Apple took the app out of the App Store because it violated terms of service that ban apps that risk "physical harm to people." The app relaunched as "Citizen" in 2017, with Frame saying that the original name "distracted from our mission" and that people should not take the law into their own hands. They should use Citizen to avoid crime rather than fight it.

Advertisement

In practice, Citizen is an app that experts say fuels paranoia and a fear of one's neighbors and surroundings by reporting "suspicious" people. Many of the incidents reported on the app are about people experiencing homelessness, for example. Citizen says it "does not report on suspicious people, nor does it report on people experiencing homelessness. Citizen reports on safety incidents such as car crashes, fires, and searches for missing people." However, many of the incidents do indeed relate to incidents involving people experiencing homelessness and many of the comments are overtly about homelessness.

"It plays into people’s anxieties and fears and magnifies people’s fears of the other and who and what they think should not exist in their neighborhood or their area," Chris Gilliard, a research fellow with the Technology and Social Change Research Project at Harvard Kennedy School’s Shorenstein Center, said. "As we’ve seen, that often means people who don’t look like them." 

One former Citizen employee told Motherboard that a portion of the app's user base is "insanely racist, which comes out in comment sections that are especially vile even by the standards of internet comment sections." Citizen does moderate comments, but "two people having an argument about whether or not someone’s comment is racist drives engagement," the source added. A hacker recently scraped a wealth of information from Citizen, including user comments that repeatedly use the N-word, according to a screenshot provided by the hacker. Some of these were deleted by Citizen, but racist comments are regularly posted on incidents.

Advertisement

Citizen incentivizes both its employees and the public to create incidents because they are the core currency of the app and what drives user engagement, user retention, and a sense of reliance on the app itself. The scrape of Citizen data published by the hacker earlier this week and shared with Motherboard shows at least 1.7 million incidents in the United States.

incident_graph_final_1.png

The weekly incidents on Citizen, using data scraped by the hacker. Image: Ishaan Jhaveri, Computational Research Fellow, Tow Center for Digital Journalism, Columbia University.

Workers have been measured by how many Citizen users see incidents they covered, how many reports they produce, and how quickly they do so, multiple former employees told Motherboard.

"It’s basically an anxiety sweatshop," a Citizen source said. "On days when things are 'slow,' they relax the standards around incidents because a dip in incident count is really bad," they added. The company sends congratulatory emails announcing which analysts reported the highest number of incidents, another source added.

This results in Citizen warning users about "everything," according to one former employee. This includes lost dogs, minor car crashes, unsubstantiated reports of gunshots, and domestic incidents, they said. This week in Los Angeles, incidents ranged in severity from "assault" to "gunfire" to "two men brawling" to "injured bird," "firefighter activity," and "crowd gathered."  

“In a healthy society we are typically not incentivized to sensationalize mundane events and code them as crime. I can’t help but think it plays into people’s anxieties and fears and magnifies people’s fears of the other,” Gilliard said. “What’s really dangerous is the ways they’re starting to serve as infrastructure, where people start to feel like they have to use them to maintain society and order.”

Advertisement

A former employee added, "They don’t much care about the accuracy or the usefulness of the information they put out, they just want to push as many notifications to create that feeling of vulnerability that leads people to the subscription services." 

Another former employee said that although fear is an aspect of the app, it isn't the only one: Notifications about fires get a lot of engagement, even if the danger to other people is not imminent, "because the videos get crazy." They added that Citizen sometimes acts as a source of entertainment for users. "People like to read about and watch videos of incidents around them," they said. 

Even Frame, the company's CEO, acknowledges Citizen's bombardment of notifications. "We send so many dumb notifs," Frame said in one of the Slack messages obtained by Motherboard. 

andrew_frame.jpeg

Andrew Frame. Image: Steve Jennings/Getty Images for TechCrunch.

There is a formal process for creating incidents, one of the sources explained. Medical calls are not reported and domestic violence incidents are recorded at an intersection rather than an address, they said. But "the guidelines evolve, and there is definitely disagreement internally over the guidelines." In the Palisades incident, Frame said he "overrode the policy we have for Mission Control" in official notes from a company all-hands meeting.

A Citizen spokesperson said in a statement that "We continue to work to improve the relevance and frequency of notifications. In addition, we are focused on reducing the reach of notifications about violent incidents, and increasing the reach of notifications about incidents such as missing people or pets being reunited with their families—we could all use some more good news."

Advertisement

To help with the deluge of incidents that Citizen creates, it has outsourced work to CloudFactory, a data processing service, three sources told Motherboard. CloudFactory uses workers based in Kenya and Nepal, according to CloudFactory's website. Citizen's use of CloudFactory has not been previously reported. CloudFactory did not respond to a request for comment.

Citizen hires "central analysts" to listen to police scanners and then enter information into Citizen so users can receive a notification about an incident close to them. Now some of those jobs are being filled by CloudFactory. 

Citizen's outsourcing of labor overseas is making U.S. employees nervous they may lose their jobs, one added. They added that the U.S. workers train the CloudFactory workers. Because Citizen is focused on specific cities in the United States, remote workers at CloudFactory are inherently not on the ground, though Citizen says its own "analysts" review incident updates.

A Citizen spokesperson told Motherboard in an email that "As the company grows, we are working with CloudFactory to augment our team. All incident updates and alerts continue to be reviewed by our 24/7 team of Citizen analysts."

Citizen believes that the more people who click into the app, the more users sign up, a source with knowledge of the company's operations said. So, like many other apps and tech companies, Citizen experiments to see where the optimal engagement may be.

Advertisement

But Citizen presents itself as a public-safety service while also trying to increase engagement. This creates a fundamental friction that, at best, ends with a barrage of useless push notifications. At worst, it ends with the CEO putting a bounty on an innocent person's head.

"It’s basically an anxiety sweatshop."

Ultimately, monetizing its user base has also led Citizen to test a product where users could order on-demand help from a private security service. This month, Motherboard reported that a Citizen-branded vehicle was driving around Los Angeles. Leaked emails showed the vehicle was part of a pilot working with Los Angeles Professional Security, a local security company whose CEO wants the power to arrest people and take them to jail. Citizen told CBS on Wednesday that the trial with Los Angeles Professional Security is now over.

The emails also showed Citizen is testing the program with well-known security firm Securitas, and claimed that high-level members of the LAPD said the product could be a game changer. A Citizen spokesperson told Motherboard at the time that the vehicle was part of a trial to test a service for users that, for example, may want an escort to walk them home. But details from internal emails already published by Motherboard explain that Citizen believes it can help with property crime.

Advertisement

This move explicitly into the private security space is directly linked to Citizen's potential plans for monetization. A Citizen product roadmap document obtained by Motherboard lists "Paid Private Response" under a section titled "$$$." That section  also mentions "Insurance Perks," "Open Hardware Platform," and "Send First Responders.". One former employee mentioned that OnAir, the broadcasting service used during the manhunt, may be turned into a premium content service, and the roadmap also mentions "Live Video" under the "$$$" section.

A Citizen spokesperson told Motherboard in an email that the company's monetization plan fits with its vision of being a global safety network of people protecting each other, but did not specify what that would involve. "We will not serve ads or sell user data," the spokesperson said.

"I never thought Citizen would go this far," one of the sources said. "I didn’t anticipate the ways they would circumvent the police and go that much further," referring to the private security force Citizen piloted in Los Angeles.

While Frame was instructing his team to up the stakes of the search for the alleged arsonist, the public faces of the manhunt were two people presenting Citizen's live broadcast. As part of its OnAir product, Citizen essentially runs a pseudo-cable news TV show, with presenters reading out tips they've received from the public, speaking to people on the ground, and, in the case of the wildfire, reminding users that Citizen was offering a bounty for information leading to the arrest of the suspect.  

Advertisement

"We mobilized the community, people in the Palisades area and surrounding. People have sent in so many tips saying that this is the guy," a woman named Kris, the main host of the broadcast, said. (A Motherboard employee in Los Angeles repeatedly got push notifications during the stream and took notes at the time.) "We had a bunch of people send in the photo that this is a known arsonist in the area. They know this guy, this is where he hangs out. While it's not official, so many people have thrown out this guy's name. He goes by the name [redacted by Motherboard] and it seems like everybody in the community knows this is the guy. So we are offering a $30,000 reward."

Toward the end of the night, Kris had been joined by Prince Mapp, Citizen's head of community. He pointed out the stakes: "When have you seen 860,495 people committed to finding one person? We have mobilized a city to bring one person to justice," he said. "Look for [the person's name]. Look for him. Family members of [the person's name]. He wasn't just brought on this world by himself, we need your help. We need you to help us contact him and identify where he is. We need the scent of his clothing. We need this man off the street so we can stop burning the city of Los Angeles […] This person is the devil and we need to get him off the street. We need to get our city back in order."

Throughout the night, these broadcasters were being coached behind the scenes in another Slack room that Andrew Frame seemingly didn't participate in much that night: "Kris, keep repeating the $10k reward. 'LET'S FIND THIS ARSONIST,'" one employee wrote. "REWARD. MONEY MONEY MONEY," they added later. "Don't stop mentioning reward for the next 7 minutes."

Advertisement

The model for the Palisades fire response, according to official notes posted in a Slack room from an all-hands meeting about the incident, was similar to something that happened in the Bronx earlier this month, when an autistic boy called Jeremiah went missing. "We had this really strong moment with Jeremiah," the notes quote Frame as saying. "This seemed like a perfect opportunity to use OnAir."

In that case, Citizen users went to Jeremiah's family's house and started filming, and Citizen itself started one of its OnAir broadcasts, where, similar to the bounty case, hosts read out incoming tips and interview users on the ground.

"There had been sightings of him running away from users trying to approach him," a source with knowledge of Citizen's operations said. 

Eventually, two users found Jeremiah at a Target and he agreed to go with them. Jeremiah got in their car and the people took him home. 

"They kept the broadcast going and had his grandma and mom on speakerphone," the source added. "People were also concerned that he got in a car with random strangers."

After the incident, Citizen uploaded a supercut of the event, with the Citizen users finding Jeremiah and him being reunited with his family. The company added hopeful background music to the footage. A Citizen spokesperson told Motherboard in a statement that "In the last 30 days, Citizen has shared critical safety information that has contributed to at least four missing people, including a 13-year-old autistic boy, reuniting with their families."

Advertisement

What the video didn't make clear is that the people on the ground filming Jeremiah and speaking to his family were not Citizen users. They were part of Citizen's "Street Team," who go out to events and contribute footage to the app while posing as ordinary users. Two sources confirmed the Street Team's existence; Citizen doesn't publicly acknowledge the existence of this team, one of the sources said. The sources added they believe the purpose of the Street Team is to make regular users think they could get involved too, so they will start broadcasting their own footage. "Essentially for engagement," one of the people said.

jeremiah.png

A screenshot of the OnAir broadcast featuring Jeremiah. Redaction by Motherboard. Image: Citizen.

"These Street Team members were not dressed in Citizen gear and looked like regular users," the source added. A document written by the company after the incident says "When Jeremiah went with Prince and Chris into the car, users did not know that they were members of the Citizen team. As a user, this scene could be alarming. Commenters had already shared how kids with autism might go easily with strangers, and then they watched the live broadcast as he did just that. Moving forward, how can we ensure our street team is presented as a safe community alternative to police?" 

Asked about the Street Team, a Citizen spokesperson told Motherboard in an email that "From time to time, we put temporary teams in place in some of the cities where Citizen is available to demonstrate how the platform works, and to show responsible broadcasting practices—similar to how social media platforms have paid creators." Slack chats obtained by Motherboard indicate Citizen may have used a Street Team during the bounty incident as well.

Advertisement

The Citizen document about the Jeremiah case also reflected on what could be improved, asked how the app could be changed to help setup search parties for users to join, or for people to report missing persons, and acknowledged that there were significant privacy, safety, and special needs issues with how Citizen handled the live streaming of a search for an autistic child.

Externally, this was presented as a success.

"If you're watching Citizen, keep watching so we can reunite more people, and use technology for good," the narrator of the recap video posted by Citizen said

"The Jeremiah incident was spun extremely positive despite all red flags—a missing child was found live. That was how it was presented. So it was essentially fuel for them to continue searching for these big incidents that would activate users," a source with knowledge of the company's operations added. 

Shortly after, another such incident appeared: the Palisades wildfire. In contrast to how the Jeremiah broadcaster said Citizen was about reuniting people and using technology for good, during the Palisades manhunt Frame said on Slack he wanted to create a situation where criminals felt like "'This is tech closing in on you. Good luck buddy.'"

Publicly, Citizen said it was very sorry for the Palisades fire incident and for putting a bounty on the head of an innocent man. The company called the entire incident "a mistake we are taking very seriously." 

Kris and Prince, two of the public faces of the manhunt, both posted lengthy messages on Slack about what they believe happened Saturday night. Kris said she didn't know what she was getting into: "It was not immediately clear that the pursuit/reward was the angle of the OnAir. we have successfully covered wildfires before without that aspect," she said. "I will say I agreed to help before I understood this but that I take full responsibility for not backing out once it was clear I’d have to say the reward."

Prince said "I went too far and I'm willing to accept the consequences. We should not be labeled as lynchers or encourage anyone to capture. I understand that we are not a news app. We are not the cops either."

While many employees at Citizen felt the Pacific Palisades incident was a huge mistake, Andrew Frame looked at it differently. While Frame showed some contrition, he sees the bounty experiment as a "massive net win," a step on the way for his app to become a private safety network that is "going into what the government is failing to do," which is, in the company's mind, failing to keep people safe, according to his Slack response to Prince.

wildfire.jpeg

Firefighters battle the Palisades fire in Los Angeles. Image: Eric Thayer/Bloomberg via Getty Images.

"There is no consequence for taking a risk, safe environment," he said, not acknowledging the potential danger the company put the man they falsely accused in. "This product work is the future of the world ... the team came together on a Saturday and pulled off something incredibly awesome."

In a later all-hands the Wednesday after the incident, Frame admitted that the company "got a lot of things wrong Saturday," according to official notes posted on Slack and obtained by Motherboard. But "even though this feels really bad on the outside, it's not nearly as bad as it may feel. One of our investors wants to assuage everyone's concerns," Frame said. "The investors have never been more excited."

"For me, this is a cultural milestone," he said, adding that the team had devised a new strategy: "The next 50-100 OnAir stories will be heartwarming ones because we will do this responsibly." When asked questions about the meeting, a Citizen spokesperson told Motherboard in a statement that "We don’t comment on anecdotes from internal meetings that are taken out of context."

Frame said at the all-hands that he is still performing a manhunt for the person Citizen falsely accused, but this time in order to apologize. 

"We need to find this person and we are actively looking to find him. We are not done when it comes to this person," notes from the all-hands say. "Andrew [Frame] said they are working on that and this has the chance to turn into a very happy moment."