It seems like not a week goes by when there’s not some kind of “gun research” being touted by the media. A few years ago, Cam wrote a piece going into the fact that so many of them are absolute garbage, and the ones since then aren’t any better. There’s been no recognition of that fact by the media at all, in fact.
Why is that?
First, we need to understand how the media deals with studies in the first place.
A decade ago, a science journalist named John Bohannon decided to play a little trick on his profession. An actual PhD, unlikely many science journalists, Bohannon actually understood what good research looked like.
It was prompted by a couple of German TV journalists who wanted to expose the “junk-science diet industry.” Bohannon had already called out pay-to-play journals, so he expected the call.
Then they got started.
Onneken and Löbl had everything lined up: a few thousand Euros to recruit research subjects, a German doctor to run the study, and a statistician friend to massage the data. Onneken heard about my journal sting and figured that I would know how to pull it all together and get it published. The only problem was time: The film was scheduled to be aired on German and French television in the late spring (it premieres next week), so we really only had a couple of months to pull this off.
Could we get something published? Probably. But beyond that? I thought it was sure to fizzle. We science journalists like to think of ourselves as more clever than the average hack. After all, we have to understand arcane scientific research well enough to explain it. And for reporters who don’t have science chops, as soon as they tapped outside sources for their stories—really anyone with a science degree, let alone an actual nutrition scientist—they would discover that the study was laughably flimsy. Not to mention that a Google search yielded no trace of Johannes Bohannon or his alleged institute. Reporters on the health science beat were going to smell this a mile away. But I didn’t want to sound pessimistic. “Let’s see how far we can take this,” I said.
The Con
Onneken and Löbl wasted no time. They used Facebook to recruit subjects around Frankfurt, offering 150 Euros to anyone willing to go on a diet for 3 weeks. They made it clear that this was part of a documentary film about dieting, but they didn’t give more detail. On a cold January morning, 5 men and 11 women showed up, aged 19 to 67.
Gunter Frank, a general practitioner in on the prank, ran the clinical trial. Onneken had pulled him in after reading a popular book Frank wrote railing against dietary pseudoscience. Testing bitter chocolate as a dietary supplement was his idea. When I asked him why, Frank said it was a favorite of the “whole food” fanatics. “Bitter chocolate tastes bad, therefore it must be good for you,” he said. “It’s like a religion.”
The study was intentionally constructed poorly. They simply checked a lot of different factors, found something that was statistically significant, had it published, and then sent out a press release written with the help of a PR specialist.
The media chomped on it.
Headlines across the world touted the health benefits of dark chocolate as a weight-loss treatment. Almost none reached out to Bohannon to talk about the study critically. They just ran with it.
Now, diet studies produce headlines that lead to traffic. No one is going to dispute that, and if they’re simply reporting what a researcher did, one might argue they’ve done their jobs.
Except these are science journalists who, at least in theory, should know better. Bohannon sure does, so why wouldn’t the others? Even if they’re not scientists themselves, they should at least be better versed in the subject than their colleagues who focus on other areas of news.
They weren’t, though.
They ran with it.
Which now brings me back to gun research for a moment.
See, diet studies might produce traffic, but debunking the research likely would, too. There’s a certain laziness involved in the chocolate hoax, but now let’s hand research to journalists who aren’t really exposed to scientific research all that often. Let’s also make sure the findings conform primarily to their own political biases.
Do you think any of them are going to look at this so-called research critically? Do you think they’re going to delve into the methodology and note the issues?
Sure, many do reach out to outside parties who are described as experts–something that didn’t happen with the chocolate hoax study–they tend to reach out to heavily biased sources like Everytown or Giffords. They talk to other researchers who produce the exact same tripe and thus have no reason to suggest methodological malpractice.
And understand, thanks to X/Twitter, we know precisely what these journalists think politically, and alarmingly few are remotely open to the pro-gun rights way of thinking, much less share it.
So, all those garbage studies noted a few years back? The media still touts them as definitive. They’ve never looked at the research critically in part because they don’t know how, and also in part because it conforms to their own beliefs in such a way that they wouldn’t think to do so.
If no one would try to debunk junk diet science that was the result of an intentionally badly constructed study on something that’s relatively trivial, why would they ever do anything to undermine their preferred narrative?
Read the full article here
