Widening the circle of scrutiny
Dr. Malički added: “A published paper should not be considered the ‘final word,’ even if it has gone through peer review and is published in a reputable journal. The peer-review process means that a limited number of people have seen the study.”
Once the study is published, “the whole world gets to see it and criticize it, and that widens the circle of scrutiny.”
One classic way to raise concerns about a study post publication is to write a letter to the journal editor. But there is no guarantee that the letter will be published or the authors notified of the feedback.
Dr. Malički encourages readers to use PubPeer, an online forum in which members of the public can post comments on scientific studies and articles.
Once a comment is posted, the authors are alerted. “There is no ‘police department’ that forces authors to acknowledge comments or forces journal editors to take action, but at least PubPeer guarantees that readers’ messages will reach the authors and – depending on how many people raise similar issues – the comments can lead to errata or even full retractions,” he said.
PubPeer was key in pointing out errors in a suspect study from France (which did not involve Surgisphere) that supported the use of hydroxychloroquine in COVID-19.
A message to policy makers
High stakes are involved in ensuring the integrity of scientific publications: The French government revoked a decree that allowed hospitals to prescribe hydroxychloroquine for certain COVID-19 patients.
After the Surgisphere Lancet article, the World Health Organization temporarily halted enrollment in the hydroxychloroquine component of the Solidarity international randomized trial of medications to treat COVID-19.
Similarly, the U.K. Medicines and Healthcare Products Regulatory Agency instructed the organizers of COPCOV, an international trial of the use of hydroxychloroquine as prophylaxis against COVID-19, to suspend recruitment of patients. The SOLIDARITY trial briefly resumed, but that arm of the trial was ultimately suspended after a preliminary analysis suggested that hydroxychloroquine provided no benefit for patients with COVID-19.
Dr. Malički emphasized that governments and organizations should not “blindly trust journal articles” and make policy decisions based exclusively on study findings in published journals – even with the current improvements in the peer review process – without having their own experts conduct a thorough review of the data.
“If you are not willing to do your own due diligence, then at least be brave enough and say transparently why you are making this policy, or any other changes, and clearly state if your decision is based primarily or solely on the fact that ‘X’ study was published in ‘Y’ journal,” he stated.
Dr. Rao believes that the most important take-home message of the Surgisphere scandal is “that we should be skeptical and do our own due diligence about the kinds of data published – a responsibility that applies to all of us, whether we are investigators, editors at journals, the press, scientists, and readers.”
Dr. Rao reported being on the steering committee of the National Heart, Lung, and Blood Institute–sponsored MINT trial and the Bayer-sponsored PACIFIC AMI trial. Dr. Malički reports being a postdoc at METRICS Stanford in the past 3 years. Dr. Krumholz received expenses and/or personal fees from UnitedHealth, Element Science, Aetna, Facebook, the Siegfried and Jensen Law Firm, Arnold and Porter Law Firm, Martin/Baughman Law Firm, F-Prime, and the National Center for Cardiovascular Diseases in Beijing. He is an owner of Refactor Health and HugoHealth and had grants and/or contracts from the Centers for Medicare & Medicaid Services, the FDA, Johnson & Johnson, and the Shenzhen Center for Health Information.
A version of this article first appeared on Medscape.com.