The meat industry comes up with a perversion of evidence-based medicine.
A series of articles published in the Annals of Internal Medicine culminated in a recommendation suggesting people keep eating red and processed meat. Nutrition researchers blasted these articles. The chair of the nutrition department at Harvard called it “a very irresponsible public health recommendation,” and the past Harvard nutrition chair was even less restrained. “It’s the most egregious abuse of data I’ve ever seen,” said Walter Willett. “There are just layers and layers of problems.” Let us start to pick through these layers.
The first of “several serious weaknesses” was that the analyses and recommendations were largely based on the so-called GRADE criteria (Grading of Recommendations, Assessment, Development, and Evaluation), which I talked about in my last blog. The authors erred by applying the GRADE tool, since that was designed for drug trials.
As I discuss in my video Failing GRADE for Annals of Internal Medicine Meat Studies, GRADE automatically gives observational studies “low- or very-low” scores for “certainty of evidence,” which is exactly what you want when you’re evaluating evidence from drug trials. You want a randomized, double-blind, placebo-controlled trial to prove the drug’s risks and benefits. “However, the infeasibility for conducting large, long-term randomized clinical trials on most dietary, lifestyle, and environmental exposures makes the criteria inappropriate in these areas,” since it would involve controlling people’s diet every day and tracking them for decades.
“You can’t do a double-blinded placebo-controlled trial of red meat and other foods on heart attacks or cancer,” the Harvard nutrition department chair said. “For dietary and lifestyle factors, it’s impossible to use the same standards for drug trials.” Imagine directing one group of people to smoke a pack of cigarettes every day for 20 years to prove that cigarettes cause lung cancer. And how could it be double-blind—have the control group smoke placebo cigarettes?
Yet, in the meat papers, they were downgrading studies due to “lack of blinding.” But, come on—in nutritional trials, how can people be blinded to what they’re eating? GRADE is just the wrong tool for diet studies. In fact, the authors admit that the reason their recommendations differ from all the others is that other guidelines haven’t used the GRADE approach. And the reason is “we can’t randomize people to smoke, avoid physical exercise, breathe polluted air or eat a lot of sugar or red meat and then follow them for 40 years to see if they die,” said a Stanford nutrition scientist. “But that doesn’t mean you have no evidence. It just means you look at the evidence in a more sophisticated way.” Besides, alternative approaches to GRADE exist—for example, NutriGrade—that were specifically developed to evaluate evidence from studies of nutrition and lifestyle factors.
So, are the authors’ appeals to standards of evidence motivated by a genuine interest in getting to the bottom of it, or just by a desire to advance the industry’s financial interests, as the same lead author had done previously at the behest of soda and candy companies? “The tool he employed in his meat and sugar studies could be misused to discredit all sorts of well‑established public health warnings, like the link between secondhand smoke and heart disease, air pollution and health problems, physical inactivity and chronic disease, and trans fats and heart disease.” Industries could use it to sow doubt in any field where randomized controlled trials aren’t feasible, like climate change. (How would a placebo planet even work?) Strictly adhering to GRADE guidelines could even be used to question the link between smoking and lung cancer.
People can’t be randomized to smoke, but can’t they be randomized to quit? A randomized controlled trial studied the effect of advising middle-aged men to stop smoking. Of those randomized to receive advice to quit, 13.7% died within the study window, compared with 12.9% in the control group who got no special instruction. In other words, it didn’t work. “Disappointingly, we find no evidence at all of any reduction in total mortality,” the researchers concluded. Wait a minute, so is smoking not bad for you after all? Of course not. Does anyone see the fatal flaw? They didn’t randomize people to quit—they randomized people to simply get advice to quit. It’s not like they could lock people in a room for a few years. At the last follow-up, those who received advice to quit were smoking an average of 8 cigarettes a day, compared to 12 cigarettes a day for those in the control group; so, it’s no surprise there was no difference in mortality since there was hardly any difference in the amount they were smoking. It’s the same with diet.
There have been massive randomized dietary trials—the Women’s Health Initiative and the Multiple Risk Factor Intervention Trial, for instance—that wasted hundreds of millions of dollars because people just flat-out failed to follow the dietary advice. So, the groups ended up eating similar diets, and they had similar disease outcomes, just like the randomized smoking-advice trial. It’s not like the failure was due to inexperienced investigators; these trials were conducted by some of the “very best research teams who invested enormous efforts to achieve their goals.” But it shows you can’t really run decade-long randomized trials that require changes to eating behavior. People just won’t do it. Randomized controlled trials couldn’t even show an effect of smoking on mortality, which is pretty remarkable considering that smoking is one of the most powerful known risk factors in the world.
So, basically, the foregone conclusion of putting any kind of junk to the test in this manner would echo the new meat report’s conclusion that people should eat whatever they want. It’s like a hijacking of evidence-based medicine. Of course, we want the best evidence possible, but the whole process is now being “manipulated and misused to support subverted or perverted agendas.”
“It’s very exciting and attention-grabbing to say there’s no need to reduce meat intake. It’s less exciting to say we reviewed studies to evaluate the validity using a system not meant to evaluate the validity of these studies, and what we found is nothing.” When asked if doctors can advise people whether a salad is healthier than a bowl of sugar, one of the senior co-authors of the meat papers responded that physicians should tell people that “the quality of evidence is low, so it depends almost entirely on their preferences.”
“When GRADE criteria do not allow us to strongly recommend against smoking a cigarette with your bowl of sugar, we believe that alternative grading systems are preferable.”
Doctor’s Note
This is the fourth in an eight-part series on how industries impact dietary and health guidelines. The first three blogs covered what happened when the 2015 Dietary Guidelines committee recommended reducing sugar consumption and introduced the GRADE approach to evaluating clinical guidelines:
The next four parts in the series are in the related posts below.
