Scientists are forcefully challenging a recent study that claimed to identify a rare genetic mutation that sharply boosted the risk of multiple sclerosis. The critics cite calculation errors and say they have been unable to replicate the findings — and question why the original paper was ever published in a top journal.
The withering assault has dashed hopes that the study might quickly lead to new drugs for MS and also raised questions about how such critiques are handled: like more and more journals, the one that published the paper does not run letters to the editor, making it harder for scientists to see that the claim has been hotly disputed.
As soon as the MS claim was published in June, outside scientists expressed doubts to STAT that there was really a link between the newly discovered mutation and the disease. Since then, other experts have dug into the study in greater detail, tried to replicate its findings using a database of thousands of MS patients, and checked the original paper’s statistics. In each case they concluded there is no link between the rare mutation and MS.
The study — which reported that the mutation raised the risk of MS by a whopping 70 percent — was led by Carles Vilariño-Güell of the University of British Columbia and published in the respected journal Neuron. It fingered a gene called NR1H3, finding that a rare mutation silences other genes involved in regulating inflammation or other aspects of immunity. As a result, inflammation runs wild, and the immune system destroys the protective myelin sheath on neurons, crippling their ability to carry signals needed for movement and other functions.
The purported mutation-MS link led to hopes that correcting the mutation or its downstream consequences might treat or even cure MS, which affects some 2.3 million people worldwide.
Within days, experts on MS and genetics began scrutinizing the claim.
Daniel Weeks, a professor of human genetics and biostatistics at the University of Pittsburgh, said an “extremely straightforward” analysis showed that some crucial statistical calculations in the paper were wrong. Doing them correctly “dramatically changes the strength of their claim,” he told STAT. “It’s no longer statistically significant.” That is, the association between the mutation and MS had a good likelihood of being due to chance.
Chris Cotsapas of Yale University and the Broad Institute, who studies the genetics of immune-related diseases such as MS, went further, looked for the mutation reported in the Neuron study but in different MS patients and healthy controls. Using a database maintained by the International MS Genetics Consortium, which holds information on 13 times more people than Vilariño-Güell studied, Cotsapas identified 31 people who have the mutation and also MS (out of a total sample of 32,852 MS patients in the database) and 31 who have the mutation but not the disease (out of 36,538). That is, the mutation was hardly more common in MS patients than healthy people.
“That tells us there is no association” between the rare NR1H3 mutation “and MS risk,” Cotsapas said, echoing the conclusion of a paper he posted to the online preprint site BioRxiv. “We found no evidence to support their claim.”
The third strike against the paper’s conclusions came from DNA sequence data gathered by the Exome Aggregation Consortium (ExAC). The NR1H3 mutation is seen in 21 of the 60,706 genomes in this database, Eric Minikel and Daniel MacArthur of the Broad found. That is about as frequent as the mutation appears in the people analyzed for the Neuron paper. Only if a mutation is more common in people with a disease than in those without it does it make sense to conclude that the mutation causes the disease, the Broad scientists noted. The mutation “contributes extremely weakly, if at all, to MS risk,” they wrote in an online comment on the biomedical journal portal PubMed.
If the NR1H3 mutation turns out not to cause MS, it wouldn’t be the first time a claimed link between a gene and a disease crashed and burned. Last week, another analysis of ExAC data concluded that 43 genes once linked to heart disease were likely false positives, too. But the flaws in the Neuron paper were so apparent, said Cotsapas, that the decision to publish it represents “a failure of peer review.”
Vilariño-Güel stands by his conclusion, he told STAT. He doesn’t dispute that a mutation that raises the risk of some disease should be more prevalent in patients than in healthy people, the opposite of what the ExAC and MS consortium analyses found. But he argued that while such DNA data, from the general population, “can identify common genetic variants which increase the risk of disease,” they can miss links to very rare mutations like NR1H3. Studying only DNA from families of MS patients, as he did, can reveal such links, he said. He also argued that some apparently healthy people in the MS consortium’s database might eventually develop the disease.
“I do believe [the association between the mutation and MS] is correct, although it would be nice to have an independent replication,” Vilariño-Güell said. “In addition to the data in the Neuron article, we have unpublished results that provide additional support” for the original claim.
One way to resolve the disagreement is to breed lab mice carrying the mutation and see whether they develop MS. Vilariño-Güell expects to have a colony of such mice by early next year, letting his team run experiments to determine whether the mutation really does cause MS.
In the meantime, journals’ policies make it hard for interested scientists to know what’s going on. Fewer and fewer journals run letters to the editor, said Pittsburgh’s Weeks. As a result, when readers access the disputed paper through the Science Direct portal or read it on Neuron’s website, there is no indication of the criticism. “The concern is, how can you correct the scientific record in a way everyone will see,” Weeks said.
Neuron declined to comment on whether there was a failure of peer review in its decision to publish the original paper or on plans to run criticisms of it.
Scientists involved in the dispute told STAT that they expect their critiques to be published soon as “Matters Arising,” Neuron’s section for such responses. In a statement, the journal said that it is “open to submissions that challenge previously published papers” and that those submissions undergo peer review, which letters to the editor traditionally didn’t. “This process can take some time,” the statement said.