The Last Word On Nothing

So many press releases land in my inbox that I don’t have time to read them all. But a recent release from the Canadian Medical Association Journal (CMAJ) caught my eye — “Treatment by naturopathic doctors shows reduction in cardiovascular risk factors. Randomized controlled trial.”

This would be big news if it were true. Naturopathic physicians are eager for respect in the mainstream, and if a randomized, controlled trial really showed that naturopathic medicine could produce measurable results, this could give naturopaths some bona fide credibility.

At this point you may be asking, what exactly is “naturopathic medicine” anyway? According to the American Association of Naturopathic Physicians, “Naturopathic physicians work with nature to restore people’s health.” Got that? They work with nature. Not to be confused with physicians that work against nature. For instance, oncologists who work to stop cancers made possible by genes that evolved in nature. Or infectious disease docs who fight parasites that have naturally co-evolved with their hosts.

Here’s what the American Cancer Society says about naturopathy. “Available scientific evidence does not support claims that naturopathic medicine is effective for most health problems.”  A letter published in Allergy, Asthma & Clinical Immunology in 2011 examined the treatments offered at 53 naturopathic clinics in Alberta and British Columbia and concluded,

A review of the therapies advertised on the websites of clinics offering naturopathic treatments does not support the proposition that naturopathic medicine is a science and evidence-based practice.

So if the CMAJ study truly showed that naturopathic medicine was effective for something, this would give naturopaths some ammunition against their critics. Alas, the study provides nothing of the sort.

The study’s authors split the study’s 246 volunteers into two groups. The control group received “enhanced usual care,” while the treatment group got seven sessions of “naturopathic care” in addition to the enhanced usual care.

What are enhanced usual care and naturopathic care? Who knows. Here’s how they’re explained in the paper:

Because a range of interventions were recommended to participants in the naturopathic group, the frequency and composition of each recommendation as well as participant adherence are not reported. We did not have direct control over the care given to the control group; thus, we did not track or report recommendations made by the participants’ family physicians.

In other words, the researchers have no idea what treatments and advice any of the patients were receiving. Naturopaths could select the recommendations they offered their patients from a menu of more than 20 different interventions. Many of these, like losing weight and exercise, are no different from what a traditional doctor might tell their patients. Others, like pomegranate juice and cinnamon, are more “alternative.”

The paper states, “This study has been peer reviewed” and it looks as though the authors simply printed the reviewers’ comments in the discussion section. For instance,

Some may perceive this as an unfair comparison and would prefer that we had asked whether the addition of, for example, 7 sessions of naturopathic care to usual care reduced cardiovascular risk compared to the addition of 7 sessions with a family physician.

The authors also state that critics

might reasonably suggest that our design was unfair and was geared toward showing a benefit in the intervention group.

But they didn’t let such shortcomings stop them from making sweeping conclusions. Their results showed that people in the naturopathy group had a modestly lower 10-year cardiovascular risk score and slightly lower rates of metabolic syndrome than people in the control group. From this, the researchers conclude:

Our findings support the hypothesis that the addition of naturopathic care to enhanced usual care may reduce the risk of cardiovascular disease among those at high risk.

No, that’s not what this study found. The findings don’t support the hypothesis, they just don’t contradict it. An editorial by CMAJ editor Matthew B. Stanbrook MD PhD, gets to the heart of the matter.

To the extent that these may have driven the observed cardiovascular risk reductions, one might say that the intervention worked because the naturopaths were, in effect, practising medicine.

There is nothing about this study that proves that naturopathy, rather than increased interactions with a physician, produced the better outcomes in the treatment group. In his editorial, Stanbrook writes,

Physicians should hold complementary medicine accountable to scientific standards equivalent to — but not higher than — medicine itself. Consequently, medical journals must be open to publishing complementary medicine research that succeeds in meeting these standards. CMAJ, for one, will continue to do so.

I commend CMAJ for its willingness to publish research on complementary medicine. Scientific minds must remain open to the possibility that new evidence will support treatments that fall outside of the mainstream. But CMAJ has failed in its task to hold naturopathic medicine to scientific scrutiny. The problem with this study is not that the treatment being tested falls outside of standard practice, but that the experiment was not properly designed to test the hypothesis.

*

Photo courtesy Shutterstock.com