Ongoing debate
A 2018 registry analysis showed no additional risk for cognitive impairment in patients who underwent ECT up to 40 years after therapy. A 2018 study also showed that ECT was efficacious and cost-effective for patients with treatment-resistant depression.
However, the ECT debate continues. As reported early last year, there seems to be little common ground between clinicians who believe in the utility of ECT for depression and those who vehemently do not.
For the current review, Read and colleagues performed a Medline search for meta-analyses of the efficacy of ECT for depression. Meta-analyses were only included if they comprised placebo-controlled trials that compared ECT with SECT.
Once the meta-analyses were identified, investigators assessed their component studies. This assessment was conducted by two independent reviewers who used a 24-point quality scale developed by the authors. This scale, the authors note, combines the “risk of bias” domains of the Cochrane Handbook Risk of Bias Tool with criteria related to quality of study design and reporting, as well as several criteria specific to ECT research.
The two reviewers were blinded to each other’s ratings. Interrater differences were resolved collectively.
The literature search yielded 83 potential articles; after exclusion criteria were applied, 14 remained. Three of these articles were literature reviews, one discussed different types of statistical analyses used in ECT research, one was a meta-analysis in Hungarian, one was a meta-analysis that compared ECT with SECT in a selected population of elderly people, and three focused on transcranial magnetic stimulation.
This left five meta-analyses for the review. These included from 1 to 7 of the 11 studies in question:
- Janicak et al, 1985
- Kho, van Vreewijk, Simpson, & Zwinderman, 2003
- Mutz et al, 2019
- Pagnin, de Queiroz, Pini, & Cassano, 2004
- UK ECT Review Group, 2003
The review revealed shortcomings with both the meta-analyses and the studies they included. The investigators found that the mean quality scores of the 11 studies (10.27 ± 2.45 and 11.91 ± 2.91) were not statistically different between the two raters (P = .17), whose scores were significantly correlated (P = .001).
Among the 264 total ratings, the investigators found 55 inconsistencies, which were all resolved by discussion. The mean final quality score for the 11 studies included in the review was 12.27 ± 3.20/24; eight scored 13 or less.
The results of these studies do little to support the benefits of ECT relative to SECT, the reviewers note. Indeed, only four concluded that ECT is significantly superior to SECT. Five found no significant difference, and the remaining two had mixed results.
What’s more, only two of what the investigators describe as “higher quality” studies reported follow-up data. Of these, one produced an effect size of 0.065 favoring ECT, the other showed a small benefit in favor of SECT (effect size, 0.299).
The investigators describe the five meta-analyses included in the review as “flawed,” stating that the meta-analyses “pay little or no attention to the multiple limitations of the studies they include.”
These limitations include the number of patients included in the studies (which average 37 patients); lack of a description of randomization and blinding processes; lack of patient ratings; selective reporting of findings; and the absence of assessment of patient quality of life. Furthermore, the authors note that none of the 11 studies “convincingly” demonstrate double-blinding.
Given these shortcomings, the investigators say the meta-analyses of ECT fail to prove the following:
- The short- and long-term efficacy benefits of ECT over SECT;
- Whether ECT is effective among patients who have failed other treatments for depression;
- Whether ECT prevents ;
- Whether ECT improves patients’ quality of life;
- Whether ECT is more effective in women than men;
- Whether ECT is effective in children or adolescents.
“Shoddy” research
The authors conclude that the review’s findings demonstrate the weakness of evidence that currently supports the use of ECT for depression.
“I would never have guessed how shoddy some of this research was,” said review coauthor Irving Kirsch, PhD, a lecturer on medicine at Beth Israel Deaconess Medical Center, Boston. Many of these shortcomings, Kirsch said, involve blinding and placebo effects.
“It’s not clear how you could ever run a truly blinded trial of ECT, given how pronounced the immediate side effects are,” he told Medscape Medical News. “And one of the things that’s underappreciated is the pronounced responses to placebo treatment with depression and severe depression. These can last for a very long time.”
Kirsch noted that more invasive placebo treatments, such as SECT, tend to have more pronounced effects.
“Not all placebos are created equal. We know, for example, that placebo injections are more effective than placebo pills and that placebo surgery can be extremely powerful,” he said.
The authors call for an immediate suspension of ECT until new studies address these research shortcomings.
“The doctors who perform ECT aren’t evil or stupid, they’re just ignorant of the research. What they see is very temporary benefit in some of the patients. The research suggests that about a third – half at most – get a very temporary lift in mood, which seems to be the sort of euphoria you get from mild brain injury,” Read explained.
“I have seen people who haven’t spoken or eaten for several weeks start speaking and eating, but we know that 4 weeks later, they’re going to be just as depressed as they were, or worse. And now they’re going to have brain damage as well,” said Read.
He added that physicians often don’t see long-term patient outcomes, just the immediate effect of ECT.
A “lifesaver”
No recent randomized controlled trials regarding ECT have been conducted because “no IRB [institutional review board] on planet earth will allow such a trial because of the overwhelming evidence of efficacy and the risk of anesthesia with no ECT,” noted Mark S. George, MD, the Layton McCurdy Endowed Chair in Psychiatry at the Medical University of South Carolina, Charleston.
It is a “logical fallacy” to conclude that ECT does not work because the trials were flawed, said George, who was not involved with the current review
“This is not supported by anything they have looked at,” he told Medscape Medical News. “It’s not really a scientific study when you make conclusions that aren’t based on your data, or not what you set out to do. That’s what I find egregious here.”
The way he sees it, the utility of ECT is unquestionable. “It is our most effective acute treatment for depression, and it’s our most effective treatment for suicide,” he said.
“The authors of this review don’t see the patients that I see every day, who are catatonic, who can’t eat, who are suicidal. For those people, ECT is a lifesaver,” George added.