A Brief History of the US Government Spying on Its Citizens

By Ray Downs

Nothing is ever new. The NSA’s monitoring of the internet and the phone records of millions of Americans surprised some people, and made the public as a whole fairly pissed off, but it’s not the first time US government has taken an Big Brotherly interest in its citizens. The feds have been tapping into the private lives of Americans without warrants and with the help of communication companies for nearly a century. Here are some of the more significant spying programs:

Black Chamber

Herbert O. Yardley, the head of Black Chamber, who later revealed its secrets. 

What many consider a predecessor of the NSA began in the 1920s, when a group of Army codebreakers working under the friendly-sounding moniker “Black Chamber” began a project to spy on other nations’ communications. Black Chamber’s chief was Herbert O. Yardley, considered to be one of the greatest cryptologists of all time. The idea was to monitor international telegraphs as the were sent to and from the US in search of anything that could affect national security—but first they had to get those telegraphs, so they asked the presidents of the major telegraph companies if they could see them.

Turns out, companies like Western Union were happy to comply. According to journalist James Bamford, who has reported widely on the NSA, Yardley later wrote: “After the men had put all our cards on the table, President Carlton [of Western Union] seemed anxious to do everything he could for us.”

The Black Chamber program set the template for a now-familiar story: a group is put in charge of monitoring communications in the name of national security and gains access to those communications through the cooperation of the private companies that own the communication infrastructure. It was Western Union telegrams back then; it’s Facebook messages now.

Funding for this program ended in 1929, and Yardley would go on to write a book titled The American Black Chamber that exposed the group’s activities. Quite possibly the first whistleblower in the American intelligence community, he is now disparaged on the NSA website and official histories, where he's described as a “disgruntled” employee who wrote the book because he was “unemployed and accustomed to luxury.” The NSA calls the book itself a “monumental indiscretion” that put America in danger. But this exposure didn’t stop future spying programs.

SHAMROCK

Screencap via The Daily Show

Operation SHAMROCK began as an Army program in 1945. During World War II, the Army had legal access to the cables of the three major communication companies of the day (RCA Global, ITT, and Western Union). But with the war over and won, that access was considered illegal, though apparently the government didn’t want to just stop listening in. According to The Lawless State: The Crimes of the US Intelligence Agencies, the Army Signals Security Agency asked the companies to continued to let them monitor international cables. The companies agreed, but there was a twist. As the authors of The Lawless State wrote, “The government apparently never informed the cable companies that its activity was not limited to foreign targets but also analyzed and disseminated the telegrams of Americans.”

In 1952, the NSA was created and took over SHAMROCK and soon the agency—which the vast majority of Americans had never heard of—was intercepting 150,000 messages a month.

During the Church Committee hearings in 1975, which publicly revealed the existence of the NSA in the process of looking into misconduct of America’s intelligence agencies, Lew Allen Jr., the the NSA’s director at the time, said that Congress passed a law in 1959 that “provides authority to enable the NSA as the principal agency of the Government responsible for signals intelligence activities, to function without the disclosure of information which would endanger the accomplishment of its functions.” In other words, it could pretty much do what it liked.

The NSA used that freedom, according to a Congressional investigator who discovered SHAMROCK, to establish secret facilities in several cities, including New York, San Francisco, Washington DC, and San Antonio. In each city, NSA employees would go to the major telegraph companies and copy telegrams, with the companies’ permission but without warrants.

As Bamford describes it, agents would “bring [the telegrams] to an office masquerading as a television tape processing company. There they would use a machine to duplicate all the computer tapes containing the telegrams, and, hours later, return the original tapes to the company.”

This program, conducted illegally yet with full cooperation with the US government and the communication giants of the day, went on for 30 years. It didn’t end until 1975, when the secret finally came out thanks to the Church Committee.

MINARET

A UNIVAC system purchased by the NSA in 1963. Photo via Motherboard

Project MINARET, often described as a sibling to SHAMROCK, was a reaction to the social movements of the 1960s. This was the program the NSA used to target “domestic enemies,” a.k.a. those involved in civil rights and antiwar groups.

Around this time, the NSA had become a sort of tool for other federal agencies, like the FBI and CIA, who would supply names of “subversives” to monitor. The NSA didn’t need warrants to tap people’s phones and was more than happy to comply. The agency also worked with the Bureau of Narcotics and Dangerous Drugs (the precursor to the DEA) to target suspected drug traffickers; this was the dawning of the war on drugs and the beginning of the federal government’s obsession with using controlled substances as an excuse to watch and arrest Americans. During the Church Committee hearings, Allen often mentioned “drug trafficking” in the same breath as “terrorism” when describing the objectives of his agency.

It took the Church Committee to expose the fact that both SHAMROCK and MINARET were blatantly illegal and in violation of the Fourth Amendment. That led to the passing, in 1978, of the Foreign Intelligence Surveillance Act (FISA), the law that established procedures for surveillance of communications. After FISA, if the government wanted to spy on suspected foreign agents within the US, they had to go to a secret court and get permission.   

That may have slowed down the NSA’s surveillance programs, but it hardly stopped them.

ECHELON, Stellar Wind, and a Decade of Whistleblowers

Image via 

“The average person doesn’t have a concept of the massive capability that is available to the National Security Agency,” said former NSA analyst William Weaver in a 2007 interview with Frontline. “Forget about the idea of a guy with earphones on listening to something. That’s not what happens. The calls are being sucked up by the millions and you’re engaged in data mining.”

Programs like SHAMROCK established the NSA’s model of using communication companies’ infrastructure to spy on Americans, but project ECHELON was the beginning of the government’s compulsive need to gather as much data from as many people as technologically possible.

ECHELON has never been formally recognized by the NSA; the agency’s official history doesn’t mention it. But it has been covered by many journalists around the world, including Nicky Hager, whose work led to an investigation by European Parliament. And now that we know what we know, it’s quite possibly the midwife of the now-famous PRISM program.

According to the Federation of American Scientists, ECHELON is “a global network of computers that automatically search through millions of intercepted messages for preprogrammed keywords or fax, telex, and e-mail addresses. Every word of every message in the frequencies and channels selected at a station is automatically searched.” This far-reaching operation is reportedly as a collaboration between the US, UK, Australia, and New Zealand governments

ECHELON probably has its roots in the 70s, but could have started earlier or later, since it’s still officially a secret. What is known is that once the Patriot Act was passed in 2001, the NSA got busy becoming the world’s largest data miner—and that technology wasn’t created overnight after 9/11.

Information about the NSA’s capabilities has been coming out in bits and pieces for years. In 2005, Justice Department lawyer Thomas Tamm and NSA analyst Russell Tice blew the whistle on the spy agency by telling the New York Times about a warrantless surveillance system being used to data-mine millions of phone calls and emails. This operation had been going on since 2001.

In 2006, AT&T technician Mark Klein exposed the NSA’s massive data collection operation in the AT&T building in San Francisco—a joint venture between the telecom giant and the spy agency. The operation became widely known as “Room 641A,” named after the roomful of supercomputers the NSA built to collect and analyze phone calls and internet traffic.

Also in 2006, NSA employee Thomas Andrews Drake talked publicly about Project Trailblazer, described by the Baltimore Sun as an attempt to create a system to analyze the “2 million bits of data the [NSA] collects every hour—a task that has grown increasingly complex with the advent of the internet, cell phones and instant messaging—and enable them quickly to identify the most important information.” The bloated program cost $1.2 billion and was a gigantic failure, proving that the government's spy agencies sometimes make costly mistakes.

The NSA needed somewhere to store all this data it was gathering, of course. William Binney, a mathematician who worked for the agency, told the public exactly where that was when he gave the media information about Stellar Wind, a terrifyingly massive data-storage center in Utah where all your calls, emails, texts, and Facebook messages collected by PRISM are probably filed away.

While discussing the mass surveillance systems set-up after 2001, Binney told Wired: “[The NSA] violated the Constitution setting it up. But they didn’t care. They were going to do it anyway, and they were going to crucify anyone who stood in the way.”

Judging from the agency’s history, that should be the NSA’s official motto.

Follow Ray on Twitter: @RayDowns

More on the NSA:

Yes, the NSA Can Spy on Every US Citizen

The NSA’s Data Collection Habits Are Trickling Down to Cops and Credit Agencies

A Photo History of the NSA, from Its Once-Secret Archives

Comments