Leading up to the midterm elections in 2014, the FBI published a report that cited statistical evidence to make the claim that mass shooting incidents were dramatically on the rise in the US. The overall impression was that bloody rampages like the Sandy Hook massacre and the Colorado movie theater shooting could be expected to occur more frequently in the coming years.
Several major news outlets ran with the story, reporting without much skepticism on findings that happened to dovetail nicely with a push by the White House and Democrats to enact stricter gun control measures and boost voter turnout. You can probably guess what happened next.
As noted recently by the Wall Street Journal and others, the authors of the study — Texas State University academics J. Pete Blair and M. Hunter Martaindale — have been forced to backpedal, acknowledging that their data was "imperfect" and asserting that the media coverage of their findings "got it wrong."
In an ongoing exchange in ACJS Today, a regular publication of the Academy of Criminal Justice Sciences, Blair and Hunter have defended their work, but they still stand accused of playing fast and loose with the numbers for political ends. As others have since pointed out, going all the way back to 1970 — instead of just the narrow 2000-2013 timeframe used in the study — shows that there has been no significant increase in the rate of mass shootings.
The spat highlights several important issues about the way news is produced and consumed. For starters, you can't always rely on reports that simply parrot research. The meaning of numbers often depends on who is counting what and why. If you want to actually get completely up to speed about your world by reading the news (rather than becoming a researcher of your own), you need to read the news like a defense or intelligence analyst.
Though professional intelligence analysts at the very high end of the spectrum rely on a variety of methods, and models to ply their craft, the fundamental job of any analyst really is to spot and sidestep bullshit. Bullshit is being used there as a catchall term for information that is untrue, unreliable, or will otherwise let you down when push comes to shove. Bullshit isn't exactly the same as willful deception, misinformation, or propaganda, although it can and does include those things. Bullshit is about a narrative operating independently of any concern for truth.
"People like the flavor of bullshit, the aroma," professional skeptic James Randi explained in a 2013 interview. "It's very rare that people will stand for a complete lack of bullshit in anything." Nobody is completely resistant to the temptations of satisfying bullshit. As much as I'd like to be perfectly immune, I'm sure as hell not. You're not, either. Heck, the entire pre-Iraq War debate over WMD is the mother of all case studies in falling for bullshit.
'Taking control of your information is the first step in taking control of your world.'
There are basically two ways of reading the news and consuming any kind of information. Sometimes people read to confirm their beliefs. The other way of consuming news is to scrape it for tidbits of data and information. In practice, people usually do a bit of both.
Related: On The Line: Ryan Faith Discusses National Security and Defense
So, how can you tell if you're being spoon-fed bullshit? It starts with second-guessing yourself. If you accept or reject data, ask yourself if you're doing it because the data is accurate, or because it confirms your biases.
People tend to reflexively assume that quantitative data is axiomatically bullshit-free, but be beware that those numbers you're looking at might be "advocacy research" — purely political opinions dressed up with data.
There are some dead giveaways that someone is using numeric data to fluff up an unsupported argument. For example, an article may note something costs $10 million. But if the author chooses to write it out fully (i.e. $10,000,000), it's probably a ham-fisted attempt to gain attention by being loud, rather than letting the number speak for itself. Fiddling with the timeframe to create an artificial sense of urgency is another red flag. Saying "almost $10 million per day" sounds more panicky than "$300 million per month." And don't confuse precision for truthfulness: If someone says $9,436,834 instead of "nearly $10 million," you're likely looking at some double-barreled bullshit.
But by far the most common sign that a number like $10 million (or $9,436,834) is being used as a bullshit-delivery system is a lack of context. Is $10 million a lot? For me personally, that would be winning the lottery. But it's not necessarily a lot for the US government, or to spend on something like developing a new cancer treatment. Raw data without any context is as pointless as a lone, solitary piece of a jigsaw puzzle.
Use the "What the Hell Else Were You Expecting?" test to spot a glaring lack of context. Imagine that someone is claiming the size of the Department of Homeland Security is proof of an impending police state. Now, maybe tomorrow the black helicopters will come to impose martial law, or maybe not. But exactly how big were you expecting DHS to be? Ten guys and a bunny? DHS was created by combining departments, bureaus, and agencies from all over government. Noting its size is closer to a tautology than a tipoff about an imminent police state.
But if people love data, they're absolute suckers for stories and narratives. Because bullshit is so often associated with confirmation bias, visceral satisfaction at hearing some bit of news should be your first warning sign that your natural BS immune system may be under attack. It doesn't necessarily mean you've swallowed some baloney, but it should set off alarms. The more you really, really want the news to be true, the more likely you are to accept bullshit as fact.
Also pay attention to how authors portray their villains. If you see someone characterizing food service contractors on a military base as a "private army" or "soldiers of fortune," then odds are an author is vilifying. No matter how rough food service gets, catering a war isn't the same as a casting call for bloodthirsty guns-for-hire.
'The more you really, really want the news to be true, the more likely you are to accept bullshit as fact.'
Vivid personal accounts can be so compelling that unreliable narrators seldom miss a chance to exploit them. One classic example is the use of food as an archetypical example of waste and fraud in the military. Living on cold MREs in the field — rather than getting some fancy chow on base — is hard on morale. But it's an entirely different matter to take the fact that king crab was served on base (presumably to improve morale) and frame that as incriminating evidence. At the minimum, it means the writer lacked either the expertise to understand or the curiosity to investigate what they were looking at.
Similarly, watch out for false equivalencies. For example, some have argued that contractors shouldn't handle classified data. Apparently government employees are somehow more trustworthy, even though they both go through the same security screening procedures and background checks. Certainly, Edward Snowden's disclosures have provided a powerful argument on that front. But trying to make that argument with Chelsea Manning is another thing altogether. Mostly because Manning was an enlisted US soldier, not a contractor.
Slipping up on that fundamental point is probably a pretty good clue that the author will also skip over the fact that some of the biggest US security breaches have involved government employees, such as FBI agent Robert Hanssen and CIA analyst Aldrich Ames, both of whom spied for the Soviet Union. Those cases deserve mention when discussing the trustworthiness of government employees.
Beyond the examples given above, there are a thousand and one other tells that should set off your bullshit detector. This small sample should be enough to give you a better feel for flavor and aroma of bullshit. And that's more than enough to get started.
For the most part, it's easier and more useful to pay attention to which individuals are serial purveyors of bullshit than getting hung up on institutions. If you catch a slip, an exaggeration, or some bullshit on the margins, you don't need to go nuts; some reasonable caution will probably do. But if that particular source turns out to be a non-stop purveyor of industrial-strength, weapons-grade bullshit — believe at your own risk.
Cutting through the bullshit is how you get from just being a member of the news audience to analyzing information. Becoming an active analyst rather than a passive consumer of news changes the power dynamic inherent in traditional media. Taking control of your information is the first step in taking control of your world.
Follow Ryan Faith on Twitter: @Operation_Ryan
Photo by Duncan Hull/Flickr