Before we begin, there is something I need to disclose: I am a scientist by training and received my PhD in microbiology and immunology, centered on nutritional immunology.

Whew! Real science credentials are not always popular in conversations about food, so I’m relieved to get that off my chest. Now that the cat is out of the bag, let’s dive into the real meat and potatoes: junk science.

Nothing gets me more riled up more than this. Broadly speaking, ‘junk science’ refers to studies that don’t adhere to the central and guiding scientific method, can’t be reliably tested or reproduced, and therefore present inaccurate claims or results.

I’m not the only one feeling fed up with questionable science. Fellow scientist and science writer John Bohannon has worked tirelessly to expose issues surrounding publishing and promoting bad (and sometimes fake!) science. Bohannon led a fascinating exposé that looked at publishing rates of a fake study in open access journals. Turns out, when Bohannon submitted his fake study to more than 300 journals, nearly 60% of the journals wanted to publish his results. That highlights some serious problems in the realm of scientific study.

Recently, Bohannon took things one step further to study media coverage of junk science. Bohannon and colleagues conducted a small clinical trial to ‘investigate’ the impact of dark chocolate on weight loss. They found that subjects on a particular diet (who also ate dark chocolate) lost weight faster than their control group. Sounds pretty great, right?

Turns out, if you read between the lines and think critically about this study, there are many obvious flaws. Unfortunately, many media outlets did not take the time to understand the study. Instead, they rapidly published stories touting headlines such as “Those Who Eat Chocolate Stay Slim!” and “Has the World Gone Coco? Eating Chocolate Can Help You LOSE Weight”.

Publishing and promoting junk science can have major implications, even for the average chocolate eater. As a scientist, I’m particularly frustrated that bad science is able to masquerade as real science. Bad science may be able to get people excited and interested in the latest ‘finding.’ Then, rigorous science, held to much higher standards, plays second fiddle because its headlines aren’t as sexy.

Here is my list of science sins that Thou Shalt Not commit in a study:

1. Fail to start with a clear hypothesis.

The hypothesis is the central tenet of a study. If a report does not state a hypothesis, researchers may have generated the results by increasing the amounts of endpoints studied, which automatically results in a finding. This tactic was performed in Bohannon’s chocolate study. 

2. Fail to use appropriate, well-defined, and reproducible methods.

Accuracy and reproducibility are central dogmas needed to produce reliable data that informs valid conclusions. If a study does not appropriately outline methods, as well as proper controls, view the results skeptically.

3. Use skewed or biased analysis of data.

Approaching data with personal biases is truly sinful in scientific practice. Real science will use ‘un-blinded analysis methods,’ meaning that they won’t know which subjects received the treatment, to ensure that the analysis is not influenced.

4. Use unclear, embellished, or untested statements when discussing results.

Grandiose language may lead to better headlines. But, if these terms are not strongly supported by results, this exaggeration can lead to promotion of false claims and conclusions. Rigorous science will not use absolute language. Good science also will consider other published studies when discussing results.

5. Fail to get peer review.

The “dreaded” process of peer review sometimes takes a significant amount of time. It can even lead to additional experiments and work. (Trust me on this one; I have been there.) But having a study undergo peer review is critical. Peer review creates authority to ensure that the research stands up to scientific scrutiny.

6. Fail to disclose conflicts of interest (COI).

Revealing any agreements between outside parties is critical to maintain and ensure that scientific integrity is upheld. The fine print: COIs do exist in science. What is really important is that those conflicts are properly disclosed. A COI does not automatically discredit the study; it is one important piece to consider.

7. Take an absolute stance and refuse to consider new evidence.

Being inflexible is a common trait of junk science. Real science embraces the totality of research, responding and changing course when new evidence and studies are published.

There are serious negative implications when junk science is accepted and touted by media and others. Hold all studies to the same standards, and ensure that the most accurate and scientifically rigorous data is presented. Because without these standards, important information and results get buried under questionable findings, slowing down real discovery and progress. And who wants that?