'Night Fury': Documents Detail DHS Project to Give 'Risk Scores' to Social Media Users

The Department of Homeland Security contracted the University of Alabama at Birmingham (UAB) in 2018 to design methods for assigning a “risk score” to potential pro-terrorists accounts on social media, as well as identifying information of interest regarding illegal opioid supply chain and disinformation efforts, according to internal DHS documents reviewed by Motherboard. The project is dubbed “Night Fury,” according to a report from the DHS Inspector General.

“The Contractor shall develop these attributes to create a methodology for developing a ranking, or ‘Risk Score,’ associated with the identified accounts. The Contractor shall develop tools to automate the identification process, documenting performance measures and metrics related to automating the identification process,” one of the documents reads. DHS said it stopped work on the project in 2019.

The news signals DHS’ continued focus on analyzing social media for a variety of purposes. These new documents come after Motherboard reported Customs and Border Protection (CBP) was using an AI-powered tool called Babel X to analyze travelers’ social media at the U.S. border. 

The Brennan Center for Justice obtained the new documents under a public records request and shared them with Motherboard. They include Privacy Threshold Analyses of the project and contracts. The research planned to involve CBP, ICE, TSA, and USCIS which would provide “cross-mission operational context,” one document reads.

“The use of automated processes to analyze social media to determine the likelihood that someone is ‘pro-terrorist’ and to assign a ‘risk score’ to individuals and groups online has echoes of a discredited Trump administration proposal called the Extreme Vetting Initiative, which would have monitored social media and the rest of the open internet to automatically flag people for deportation or visa denial based on whether they would be a ‘positively contributing member of society’ or ‘contribute to the national interests,’ as well as whether they ‘intend to commit’ a crime or act of terrorism,’ Rachel Levinson-Waldman, Managing Director, Liberty & National Security Program at the Brennan Center for Justice, told Motherboard in an email.

“As a number of experts in machine learning and automated decision-making told DHS less than a year before the Night Fury contract was signed,  attempting to make automated judgments about these matters is both impossible and likely to be infected with bias, as these characteristics have no concrete definition, much as there is no definition of being ‘pro-terrorist,’” she added.

One of the Privacy Threshold Analyses says UAB’s work will initially be focused on “counter-terrorism, illegal opioid supply chain, transnational crime, and understanding/characterizing/identifying the spread of disinformation by foreign entities, including the study of bot detection,” but that the methods should “scale to other DHS domains.” 

Another document says the researchers will “build next generation capabilities.” This includes developing training data sets, algorithms, and methodologies, the document adds.

More specifically, the project planned to develop methods that could identify a location without GPS metadata, such as looking for certain keywords, the document reads. The researcher also planned to track threats beyond mainstream social networks like Facebook and Twitter to other communities. DHS planned to test the methods against live events unfolding in real-time, such as a hurricane scenario, the document adds.

Another task was to create a “Facebook Group Expander,” which would automatically identify potential pro-terrorist social media accounts and Facebook Groups where pro-terrorist groups interact, one document reads. UAB would then constantly deliver lists of these accounts and related posts to DHS. UAB was tasked with doing a similar thing on Twitter, the document reads.

“In light of the reference to ‘pro-Jihad’ accounts and the long-standing targeting of the Muslim community by DHS and other federal agencies, it seems likely that this project, if implemented, would have disproportionately swept in Muslim social media users, with potentially high-stakes immigration or criminal consequences for them and even their family members,” Levinson-Waldman added. “The social media collection and analysis contemplated by Night Fury could also have an outsized impact on academics, activists and advocates, journalists, and others engaged in an online exchange of ideas about American foreign policy on terrorist groups, the validity of ‘material support’ laws, and more.”

Regarding disinformation, the DHS planned for UAB to develop automated tools that would be able to determine if a social media accounts corresponds to a single human, or whether the account was “programmatically generated to exert influence”—a bot.

The “lesser” social media networks DHS wanted UAB to also study included Telegram, Google+, VK, Ask.fm, and Zello, one document reads.

The Data Analytics Technology Center (DATC), part of the Science and Technology Directorate (S&T), is the lead organization for this project, according to the documents. UAB submitted a proposal to DHS which was accepted, the document adds. The document says UAB has “deep past experience” in identifying online content, including that related to the gang MS-13, for both private and government stakeholders. DHS awarded the contract in September 2018 with a potential award amount of $3.4 million, according to public procurement records.

In a statement, a DHS spokesperson told Motherboard "The Department of Homeland Security is committed to protecting individuals’ privacy, civil rights, and civil liberties. DHS uses various forms of technology to execute its mission, including tools to support investigations related to threats to infrastructure, illegal trafficking on the dark web, cross-border transnational crime, and terrorism. DHS leverages this kind of technology in ways that are consistent with its authorities and the law and has safeguards in place to ensure contractors or service providers also limit their work so that it is in compliance with those authorities." DHS said it stopped work on the project in October 2019 and ended the contract in December 2019.

Update: This piece has been updated to a statement and information from DHS.

Subscribe to our cybersecurity podcast, CYBER. Subscribe to our Twitch channel.