As fears of the novel coronavirus 2019-nCoV continued to spread last Friday, an inflammatory new paper appeared on bioRxiv, a preprint server, where scientists post work that hasn’t been vetted.
Titled “Uncanny similarity of unique inserts in the 2019-nCoV spike protein to HIV-1 gp120 and Gag,” the paper claimed to find similarities between the new coronavirus and HIV, the virus that causes AIDS. The use of the word “uncanny” in the title, together with “unlikely to be fortuitous” in the abstract, led some to think that the authors were suggesting the virus had somehow been engineered by humans.
The paper, from academic institutions in New Delhi, India, was critical and alarming, if true. Except that it wasn’t.
The paper was almost immediately withdrawn, but not before plenty of handwringing from researchers who complained that the appearance of such shoddy work on a preprint server without vetting by peer reviewers is precisely why the hoary old model of science publishing is better at keeping junk science out of the literature.
Support STAT: STAT is offering coverage of the coronavirus for free. Please consider a subscription to support our journalism. Start free trial today.
Except that’s not true, either. The old model has its advantages, to be sure, but it, too, is prone to the menace of pseudoscience, bad data, and other flaws — despite traditional academic journals’ army of peer reviewers. And when these publications publish bad or erroneous research, it can take months or years for the papers to be corrected or retracted — if they ever are.
In contrast, the reaction from the scientific community to the bioRxiv paper was swift. In a nutshell, commenters on bioRxiv and Twitter said, the author’s methods seemed rushed, and the findings were at most a coincidence. By Saturday morning, bioRxiv had placed a special warning on all papers about coronavirus. Later Saturday, the authors commented on their paper, saying they were withdrawing it. And on Sunday, a more formal retraction appeared: “This paper has been withdrawn by its authors. They intend to revise it in response to comments received from the research community on their technical approach and their interpretation of the results.”
All of that happened before a single news outlet with any reach covered the paper, as best we can tell. But none of it was quite fast enough for some critics. “This is why preprints can be bad,” said one scientist on Twitter. That scientist, Michael Shiloh, said he had even used bioRxiv to post preprints. “What bugs me about this preprint is that had this manuscript undergone legitimate peer review, these flaws would have led to a swift rejection and it wouldn’t be contributing to the conspiracy theories and fear surrounding this outbreak,” Shiloh continued.
History suggests that Shiloh’s confidence in peer review’s ability to suss out pseudoscience may be a bit misplaced. The fraudulent 1998 paper that set off the vaccine-autism scare was published in The Lancet, one of the world’s leading peer-reviewed medical journals. Other examples — including a paper by an intelligent design advocate questioning the validity of the second law of thermodynamics as it pertained to evolution — abound. Papers claiming a link between autism and vaccines pop up nearly every year.
And even when peer-reviewed journals do realize they’ve been had, retractions can take months or years. The Lancet took 12 years. Another journal took five years to retract a paper claiming that HIV did not cause AIDS. We could go on, and the list includes papers that have never even been corrected.
Those who claim preprint servers are dangerous because they lack peer review — bioRxiv has a perfunctory screening process — sometimes acknowledge that journals have had to speed up their game to meet the pressures of an outbreak like coronavirus, or SARS in the early part of this century. Angela Cochran, president of the Society for Scholarly Publishing, a trade group for publishers, said on Twitter: “Earlier this week, folks celebrated that coronavirus papers were popping up in preprint servers. Now there is a reminder not to use them to guide clinical practice because they haven’t been reviewed. Journals ARE reviewing coronavirus papers and getting [them] pub’d quickly.”
Kent Anderson, another publishing industry veteran, put it more bluntly: “Journals Win The Coronavirus Race.”
Publishers have been looking for ways to score points against — and shut down — preprints for at least half a century. Journals have speedily published a number of important papers on the new coronavirus already, no doubt. Publishing industry champions are often quick to say that speedy peer review does not mean sloppy peer review — even in cases that require massive corrections.
It now turns out that one of the world’s leading peer-reviewed medical journals, the New England Journal of Medicine, published a letter last week about apparent coronavirus transmission from an asymptomatic person that turns out to be wrong. Such letters to the editor are not typically peer-reviewed, or at least not as rigorously as a full study. But it’s a glaring example of how a peer-reviewed journal can end up getting things wrong. Now we get to wait and see how long it takes NEJM to correct the record.
We’ll see whether peer-review champions, who are often unwilling — with some notable and welcome exceptions — to acknowledge how slow and ineffective correction in science can be, note this apparent failure of one of their gatekeepers. Doing so might, after all, make some people question the expensive subscription deals universities agree to with publishers, as well as the article processing charges that can run into the thousands of dollars for open access publications.
Peer review can add a valuable filter. But those who work in publishing seem to be so wedded to the existing process that they can’t admit its flaws — or that it might be a good idea to also embrace preprint servers that could upend their business models. Just like in politics, maybe it’s time to agree that the publishing process is a messy one, and stop using single episodes, free of context, to score points against one’s rivals.
This article has been updated with information about a flawed report in the New England Journal of Medicine.
Are we to believe man has NEVER mutated a virus? Really? Why in the world is there such research going on all over the world? Including….Harvard (see Dr. Charles Lieber, 60, Chair of the Department of Chemistry and Chemical Biology at Harvard University, was arrested this morning and charged by criminal complaint with one count of making a materially false, fictitious and fraudulent statement).
Interesting comments on the part of the author.
It seems that questioning certain preconceived – let’s call them ‘dogmatic’ beliefs by the ‘approved’ or vetted scientists will be considered shoddy, flawed, or ‘junk’ regarding of their quality, so that even qualified reviewers of these examples will be subject to the vitriol from the ‘priests’ of the post-truth world.
Regardless of the speed of the retracted paper, exactly what analysis, experiment or research was conducted by the instant critics? Was it not they who were too rash or quick on their comdemnation? Was it not they who succumbed to considerations other than science, facts, and analyses, say perhaps to ‘political correctness’?
How lucky for Einstein that such an anti-scientific fervor did not prevail during the publication of his theory.
My point is easily supported by the author’s inclusion of the update (it should be retracted, 🙁 , perhaps): “It now turns out that one of the world’s leading peer-reviewed medical journals, the New England Journal of Medicine, published a letter last week about apparent coronavirus transmission from an asymptomatic person that turns out to be wrong. “. Maybe the specifics of the case were wrong, but to infer that such asymptomatic transmission does not occur is clearly premature given how little has been studied regarding the novel corona virus of 2019. Such transmission has been documented by genomic analysis, and this is not surprising due to the nature of virus shedding and the supposed 14 days (meta studies of corona virus shedding or incubation spread the duration up to 27 days). See Swiss Medical Weekly (https://smw.ch/article/doi/smw.2020.20212) . It is up to other researchers to replicate the analyses to support or refute these findings. (or so-called ‘scientists’ can immediately criticize these findings without even reading the text!)
Oh, how quickly the guillotine of professional excommunication lands on the necks of those who would question the religion of the ‘enlightened’ establishment, regardless of their scientific and ethical zeal. Yet science demands precisely this kind of questioning, and rejects the dogma of group-think.
It is not my point to suggest that clearly bogus methodology sometimes makes it through reviewers. Reviewing papers is tedious work, with high risk and dubious rewards, without the benefit of a higher level of reviewers, and reviewers are humans also. It is not their job to question the conclusion or to criticize statements of emotion, attitude or opinion of the authors. They are intended to review the methods and the robustness of the statistical analysis. If such robust and accepted methods and analyses should lead to a conclusion that the moon is made of cheese, so be it! In such case, maybe it’s time to rethink the scientific method!
There is objective truth, and we must search for it, while it still may be found.
Okay,
Scientific commentary about a non peer review, they listed many reputable PhDs, and credible sources in their outline of the aforementioned paper. So, looking at the page, it was internally peer reviewed by multiple authors in India.
Who is claiming the right to peer review this Indian document, names and positions, and countries of criticizing scientists, with the background to make a suitable challenges?
Next, what we’d really like to see is the laundry list of technical points made by these criticizing virologists, etc.
Only that will fly with the public, point-by-point.
I’d love to see exactly what and why other scientists can criticize, otherwise, the commentary looks like a political attack.
Next when (allegedly) a bunch of PhDs make an ad hominem
public attack against a scientific paper, it is customary to list the names of those so qualified to challenge merit, and list the key points of merit.
That would help departisanize the article, and make it much stronger, as a fact checker, because, if fact checker cannot produce either names (such as, Shilo and whom all else?) that are credible and documented opposing points of view, nor provide any factual scientific basis for their comments, then state news gets hung on a line to themselves scientifically and technically refute the research, and either keep the the article “…a faulty coronavirus paper…” because you list off actual errors in the aforementioned paper, or go back and retroactively edit language in this article that pre-supposes the Indian PhD scientists are not quacks, and use language pointing to this article being mostly correct, pending changes in a final Indian initiated peer review for the same. As it stands, this is a thoughtful article that does not yet hold water.
Exactly which assertions and statements were found to be untrue? Are there significant similarity between SARS-CoV2 and HIV when compared to the original SARS?
Seems like the dispute is over the editorializing and not the facts.
With regards to a different statement in this article about false information linking vaccines to autism. The facts are coming out slowly but surely on that topic. The CDC was recently sued by ICAN after they failed to produce studies supporting their claim that vaccines don’t cause autism from the vaccines in the first 6 months of life, through FOIA. They failed to do so and were sued, leading them to produce 20 studies that were not studies addressing the claim. The CDC conceded in court. Basically that means the studies to support those claims don’t exist. You can look for yourself on ICAN.
Looks like the pre-print server did its job. Fast access coupled with fast criticisms from the audience, that led to an equally fast withdrawal of paper.
As it turns out, the NEJM letter was correct in substance. The Koch Institute telephoning the traveller from Shanghai to solicit information about symptoms is not sound research practice, particularly when they chose to classify a backache in a long distance traveller as a potential Covid19 symptom. The entire episode smells strongly of RKI attempting to save face over their inaccurate conclusion that people who do not present with symptoms cannot transmit the virus. I don’t think anything about this should be celebrated as good science. Quite the contrary.
the paper was not withdrawn for “not being true” as you know, it was withdrawn to be vetted, because it had not yet been. It also had not been published, it was on a pre-print server, also as you know, because that information is on biorXiv (“this article has been withdrawn. click here for details.”).
The preprint proved that the virus is a bio-engineered weapon, but the mainstream media and the establishment trolls slandered the authors and prevented the truth from being widely reported. Also the authors chose not to be heroes and retracted their findings, which is totally their right, nobody want to be a martyr in 2020, especially given how few people on Earth deserve to be saved…
ROFL