Mental health treatments are hard. Most of the drugs come with side effects that range from irritating to really upsetting, and often the other options are based more on wishful thinking than proven medicine. Which makes it all the more enticing when the news seems to indicate that depression can be treated by just running a few times a week. As headlines around the world have been saying recently, a new study seems to show that running therapy is just as effective in treating depression as pharmaceutical interventions.
Whenever you read headlines like these, there’s a simple rule to remember: If it sounds too good to be true, it probably is. While exercise is probably useful to some extent in the treatment of mental health issues, there is no good evidence that running is as effective as SSRIs like Prozac and Lexapro.
The study that has hit the headlines everywhere is a new publication in the Journal of Affective Disorders. The study itself was pretty simple. The authors described the paper as a “partially randomized patient preference” study. In practice this consisted of taking people with depression or anxiety and having them pick the treatment they preferred: either standard antidepressants or running therapy. The running therapy consisted of group runs led by a trainer two to three times a week for a total of four months.
At the end of the four months, as the authors report, the two therapies “did not differ significantly on mental health outcomes” (emphasis mine). However, the running group had lost a bit of weight, and gotten a bit fitter, leading to the recommendation that “Exercise therapy … should be considered standard practice for those with depression and/or anxiety disorders.” Hence, the headlines. “Go for a Run to Beat Depression—It’s Just As Effective As Taking Medication,” declared the New York Post. The article includes this quote from the lead author of the paper: “Both interventions helped with the depression to around the same extent.”
But if you look a little more closely at this paper, the argument that running is equivalent to meds seems to fall apart entirely. Firstly, the groups weren’t truly randomized. That would involve sorting people into treatment groups, not allowing them to pick which treatment they wanted to try themselves, as the researchers did. This is a problem because it means that the people who elected to do one therapy or the other might be very different, which could explain the differences in depression results at the end of the study. Indeed, diving into the paper, it turns out that the people who chose to take meds were much more depressed and anxious than those who chose to do running therapy. Honestly, this is a bit obvious when you think about it. People who are really suffering with depression might not want to go running two or more times a week!
There’s also the small issue of that word “significantly” that appears above. The authors correctly reported that on a statistical test, there was no statistically significant difference between running and medication for alleviating depression or anxiety. But if you look at the supplementary appendix, it seems that people who took medication had about twice the improvement in depression/anxiety scores than those who went running, on average. It’s true that this effect may not be statistically significant in the test the researchers used—the fact is that the study was so small, just 141 people, that it’s difficult to know what the results mean at all. A more correct interpretation is that medication had twice the effect of running in the study, but we don’t know if that is random noise or a true effect. The sample size is simply too small to tell either way.
Overall, the study itself is extremely weak. Not only was it not randomized and quite small, the authors also preregistered a very different protocol just four years earlier. (The practice of preregistering helps hold authors to account, and allows us to ask the question: Did the researchers actually do what they say they set out to do—or did they perhaps fuss with the data until they got any interesting result, even a weak one?) In 2019, they published a protocol for the exact same study, except instead of looking at depression and anxiety scores, the main outcomes of interest were “biological aging, metabolic stress, and neurobiological abnormalities.” The protocol in 2019 also included a control group of people without mental health issues. The statistical analysis plan in the original protocol also makes no mention of the final analysis that the authors performed. This kind of discrepancy is often a sign of what’s called selective reporting or outcome switching, a practice known to substantially distort clinical trial results.
So the results from this particular trial are almost entirely untrustworthy. What does the rest of the literature show?
Frankly, this study fits right in. In general, other studies on exercise for depression are extremely low quality. According to a 2013 Cochrane review—usually considered the gold standard for evidence-based medicine—of the 39 trials published by that time on the question, just six fulfilled even the most minimal checks for quality. Looking at only those six half-decent papers, there was at best a very small improvement in depression symptoms for people who were given exercise therapy. There were very few trials looking at exercise versus medication for these conditions, and none of them were reliable. Though a more recent review, published in the British Journal of Sports Medicine, caused a hubbub earlier this year for claiming exercise should be a “mainstay approach” for managing depression, it mostly rehashed the same evidence as the 2013 Cochrane review.
Ultimately, this new study adds almost nothing new to our understanding of exercise or running as a treatment for mental health problems. To the extent that the results tell us anything, they seem to really show that medication is more useful than running two or three times a week, but even that result is very questionable.
One particular result of this trial does spring out when you read it: Of the 96 people who chose to do running therapy, half of them stopped running by the four-month mark. One potential take-home from this paper is that even for highly motivated people who choose to try running therapy for their mental health problems, many will stop the therapy soon after. Meanwhile, 82 percent of those who took medication were still compliant at the same time-point.
This is not to say that if you are anxious or depressed that running—or another form of movement—isn’t worth trying at all. If you can exercise, it’ll probably help your mental health, to some extent. Frankly, every single study we’ve ever run suggests that exercising is good for your health, physical and mental. But the evidence also seems to show that as a practical intervention, exercise has limited applicability to real people in the real world. And there’s currently no good reason to believe that it will be as effective as medication.
State of Mind is a partnership of Slate and Arizona State University that offers a practical look at our mental health system—and how to make it better.