If your doctor diagnoses you with chronic fatigue syndrome, you’ll probably get two pieces of advice: Go to a psychotherapist and get some exercise. Your doctor might tell you that either of those treatments will give you a 60 percent chance of getting better and a 20 percent chance of recovering outright. After all, that’s what researchers concluded in a 2011 study published in the prestigious medical journal the Lancet, along with later analyses.
Problem is, the study was bad science.
And we’re now finding out exactly how bad.
Under court order, the study’s authors for the first time released their raw data earlier this month. Patients and independent scientists collaborated to analyze it and posted their findings Wednesday on Virology Blog, a site hosted by Columbia microbiology professor Vincent Racaniello.
The analysis shows that if you’re already getting standard medical care, your chances of being helped by the treatments are, at best, 10 percent. And your chances of recovery? Nearly nil.
The new findings are the result of a five-year battle that chronic fatigue syndrome patients — me among them — have waged to review the actual data underlying that $8 million study. It was a battle that, until a year ago, seemed nearly hopeless.
When the Lancet study, nicknamed the PACE trial, first came out, its inflated claims made headlines around the world. “Got ME? Just get out and exercise, say scientists,” wrote the Independent, using the acronym for the international name of the disease, myalgic encephalomyelitis. (Federal agencies now call it ME/CFS.) The findings went on to influence treatment recommendations from the CDC, the Mayo Clinic, Kaiser, the British National Institute for Health and Care Excellence, and more.
But patients like me were immediately skeptical, because the results contradicted the fundamental experience of our illness: The hallmark of ME/CFS is that even mild exertion can increase all the other symptoms of the disease, including not just profound fatigue but also cognitive deficits, difficulties with blood pressure regulation, unrestorative sleep, and neurological and immune dysfunction, among others.
Soon after I was diagnosed in 2006, I figured out that I had to rest the moment I thought, “I’m a little tired.” Otherwise, I would likely be semi-paralyzed and barely able to walk the next day.
The researchers argued that patients like me, who felt sicker after exercise, simply hadn’t built their activity up carefully enough. Start low, build slowly but steadily, and get professional guidance, they advised. But I’d seen how swimming for five minutes could sometimes leave me bedbound, even if I’d swum for 10 minutes without difficulty the day before. Instead of trying to continually increase my exercise, I’d learned to focus on staying within my ever-changing limits — an approach the researchers said was all wrong.
A disease ‘all in my head’?
The psychotherapy claim also made me skeptical. Talking with my therapist had helped keep me from losing my mind, but it hadn’t kept me from losing my health. Furthermore, the researchers weren’t recommending ordinary psychotherapy — they were recommending a form of cognitive behavior therapy that challenges patients’ beliefs that they have a physiological illness limiting their ability to exercise. Instead, the therapist advises, patients need only to become more active and ignore their symptoms to fully recover.
In other words, while the illness might have been triggered by a virus or other physiological stressor, the problem was pretty much all in our heads.
By contrast, in the American research community, no serious researchers were expressing doubts about the organic basis for the illness. Immunologists found clear patterns in the immune system, and exercise physiologists were seeing highly unusual physiological changes in ME/CFS patients after exercise.
I knew that the right forms of psychotherapy and careful exercise could help patients cope, and I would have been thrilled if they could have cured me. The problem was that, so far as I could tell, it just wasn’t true.
A deeply flawed study
Still, I’m a science writer. I respect and value science. So the PACE trial left me befuddled: It seemed like a great study — big, controlled, peer-reviewed — but I couldn’t reconcile the results with my own experience.
So I and many other patients dug into the science. And almost immediately we saw enormous problems.
Before the trial of 641 patients began, the researchers had announced their standards for success — that is, what “improvement” and “recovery” meant in statistically measurable terms. To be considered recovered, participants had to meet established thresholds on self-assessments of fatigue and physical function, and they had to say they felt much better overall.
But after the unblinded trial started, the researchers weakened all these standards, by a lot. Their revised definition of “recovery” was so loose that patients could get worse over the course of the trial on both fatigue and physical function and still be considered “recovered.” The threshold for physical function was so low that an average 80-year-old would exceed it.
In addition, the only evidence the researchers had that patients felt better was that patients said so. They found no significant improvement on any of their objective measures, such as how many patients got back to work, how many got off welfare, or their level of fitness.
But the subjective reports from patients seemed suspect to me. I imagined myself as a participant: I come in and I’m asked to rate my symptoms. Then, I’m repeatedly told over a year of treatment that I need to pay less attention to my symptoms. Then I’m asked to rate my symptoms again. Mightn’t I say they’re a bit better — even if I still feel terrible — in order to do what I’m told, please my therapist, and convince myself I haven’t wasted a year’s effort?
Many patients worked to bring these flaws to light: They wrote blogs; they contacted the press; they successfully submitted carefully argued letters and commentaries to leading medical journals. They even published papers in peer-reviewed scientific journals.
They also filed Freedom of Information Act requests to gain access to the trial data from Queen Mary University of London, the university where the lead researcher worked. The university denied most of these, some on the grounds that they were “vexatious.”
Critics painted as unhinged
The study’s defenders painted critics as unhinged crusaders who were impeding progress for the estimated 30 million ME/CFS patients around the world. For example, Richard Horton, the editor of the Lancet, described the trial’s critics as “a fairly small, but highly organised, very vocal and very damaging group of individuals who have, I would say, actually hijacked this agenda and distorted the debate so that it actually harms the overwhelming majority of patients.”
Press reports also alleged that ME/CFS researchers had received death threats, and they lumped the PACE critics in with the purported crazies.
While grieving for my fellow patients, I seethed at both the scientists and the journalists who refused to examine the trial closely. I could only hope that, eventually, PACE would drown under a slowly rising tide of good science, even if the scientific community never recognized its enormous problems.
But with the National Institutes of Health only funding $5 million a year of research into chronic fatigue syndrome, it seemed like that could take a very long time.
Then last October, David Tuller, a lecturer in public health and journalism at the University of California, Berkeley, wrote in Virology Blog a devastating expose of the scientific flaws of the trial. Tuller described all the problems I had seen, along with several more. The project was a remarkable act of public service: He isn’t a patient, yet he spent a year investigating the trial without institutional support, legal backing, or remuneration.
And, at last, the criticisms gained traction.
Racaniello and 41 other scientists and clinicians published an open letter to the Lancet calling for an independent investigation into the trial and saying “such flaws have no place in published research.” Rebecca Goldin, the director of Stats.org, an organization that works to improve the use of statistics in journalism, eviscerated the trial’s design in a 7,000-word critique.
In the meantime, a Freedom of Information Act request from Australian patient Alem Matthees was making its way through the legal system.
Matthees had asked for the anonymized data necessary to analyze the study using its original standards for success, but Queen Mary University of London had refused the request, arguing that malicious patients would break the anonymization and publish the participants’ names to discredit the trial. It again cited the death threats.
The court rejected these claims a month ago, calling them “wild speculations” and pointing out that the researchers themselves acknowledged in court that neither they nor PACE participants had received death threats.
Startling results from a re-analysis
Just before releasing the data,Queen Mary University of London did its own re-analysis on the question of how many patients got better, at least a little bit. Their data showed that using the study’s original standards, only 20 percent of patients improved with cognitive behavior therapy or exercise in addition to medical care, not 60 percent as claimed in the Lancet.
And even the 20 percent figure might be misleading, because the re-analysis also found that 10 percent of participants improved after receiving only standard medical care. That suggests that 10 percent in each of the treatment groups would likely have improved even without the exercise or therapy, leaving only 10 percent who were significantly helped by those interventions.
As for the claim that 22 percent of patients who received either treatment made an actual recovery? That went up in smoke when Matthees analyzed the raw data with the help of his colleagues and statisticians Philip Stark of the University of California, Berkeley, and Bruce Levin of Columbia University.
Their analysis showed that had the researchers stuck to their original standards, only 4.4 percent of the exercise patients and 6.8 percent of the cognitive behavior therapy patients would have qualified as having recovered, along with 3.1 percent of patients in a trial arm that received neither therapy.
Importantly, there was no statistically significant difference between these recovery rates.
The PACE researchers, the editor of the Lancet, and the editors of Psychological Medicine (which published the follow-up study on recovery) all declined to comment for this article.
Simon Wessely, president of the UK Royal College of Psychiatrists, defended the trial in an email exchange with me. He argued that some patients did improve with the help of cognitive behavior therapy or exercise, and noted that the improvement data, unlike the recovery data, was statistically significant. “The message remains unchanged,” he wrote, calling both treatments “modestly effective.”
Wessely declined to comment on the lack of recovery. He summarized his overall reaction to the new analysis this way: “OK folks, nothing to see here, move along please.”
‘A classic bad study’
But it doesn’t appear that outside researchers are ready to “move along.”
After reviewing the new analysis, Jonathan Edwards, a professor emeritus of medicine at University College London said he was unconvinced that these small subjective improvements indicated the patients genuinely felt better. “They’ve set this trial up to give the strongest possible chance of there being a placebo effect that you can imagine,” he said.
“This is a classic bad study,” said Ron Davis, director of the Stanford Genome Technology Center and director of the Science Advisory Board of the End ME/CFS Project. He emphasized an additional problem: The study used such a broad definition of the disease that it likely included many patients who didn’t truly have ME/CFS at all.
“The study needs to be retracted,” Davis said. “I would like to use it as a teaching tool, to have medical students read it and ask them, ‘How many things can you find wrong with this study?’”
Retractions are rare, however, and erasing the impact of this flawed research will take much work for years to come.
After a sustained effort by ME/CFS advocates, the federal Agency for Healthcare Research and Quality, just changed its recommendation to read that there is insufficient evidence to justify cognitive behavior therapy or graded exercise. But many more public health agencies continue to point patients toward them.
And efforts to propagate this approach continue: A trial of graded exercise in children with ME/CFS has recently begun, and patients are protesting it.
Watching the PACE trial saga has left me both more wary of science and more in love with it. Its misuse has inflicted damage on millions of ME/CFS patients around the world, by promoting ineffectual and possibly harmful treatments and by feeding the idea that the illness is largely psychological. At the same time, science has been the essential tool to repair the problem.
But we shouldn’t take solace in the comforting notion that science is self-correcting. Many people, including many very sick people, had to invest immense effort and withstand vitriol to use science to correct these mistakes. And even that might not have been enough without Tuller’s rather heroic investigation. We do not currently have a sustainable, reliable method of overturning flawed research.
And rectifying PACE will take more than exposing its flaws. The lingering doubt it has cast on the illness will only be fully dispersed when we’ve finally figured out what’s really going on with the disease.
For that, we need to invest in some serious, good science. The kind I continue to love.
Julie Rehmeyer is a math and science writer. Her memoir “Through the Shadowlands,” describing the science and politics of chronic fatigue syndrome and other poorly understood illnesses, will be published by Rodale in May.
Sorry to spoil the party but some cold facts are necessaey
1. The PACE trial remains an excellent trial and a model of how to deliver a complex intervention RCT. Read the 2012 Lancet paper again. Check it against the CONSORT statement. You will see it is 100% compliant.
2. Nothing at all has emerged to alter or contradict its key findings. These are that two treatments (CBT and GET) are superior to two others (pacing and standard medical treatment). All four approaches are safe. The effects of CBT and GET are modest but still,worthwhile especially as at moment and for the last 25 years there has been nothing better
3. As others note this does not speak to the causes of CFS but only to management. For avoidance of doubt all the serious clinicians and academics who work in this area know this is genuine condition that causes much misery to patients, friends and family.
4. Clearly we need better treatments but until and unless we do, doctors can safely recommend CBT and GET to patients. Indeed they could and did before PACE – it’s findings were entirely congruent with the previous literature, just that PACE was larger. In my country (UK) there was already sufficient evidence to recommend both therapies in our NICE guidelines before PACE was published. PACE will not be retracted because there is no reason to do so, it is fully CONSORT compliant and so on, but if it had not happened the guidelines would remain the same.
I have written a commentary here on the main PACE paper outlining the basic principles of a sound RCT and why PACE more than adheres to them
http://www.nationalelfservice.net/other-health-conditions/chronic-fatigue-syndrome/the-pace-trial-for-chronic-fatigue-syndrome-choppy-seas-but-a-prosperous-voyage/
I realise that this will not be a popular post. That’s sad. But I think people need to know there is another story here and one that might explain why it is that people continue to cite the Lancet paper and why it’s results whilst not earth shattering are sound.
Finally I should add that I played a part in early days of developing CBT as a management approach to CFS, carried out one of the early RCTs and so on, and know personally all the PACE authors. I am not therefore neutral on this topic, and I continue to recommend CBT to my patients with CFS if they want it. I was not however an author on the PACE study .
Simon Wessely
I can sympathize with your spirited defense of your colleagues, but it seems to me that despite your long post, you’ve carefully avoided the main point Rehmeyer specifically asked you to address–how radically the “recovery” statistics changed when the authors weakened the criteria from their original protocol, and the failure to disclose this until forced by a tribunal. What do you make of that?
PACE is not a sound clinical trial because it lacks blinding and relies on self reported outcomes to determine treatment effectiveness. Under these conditions it impossible to distinguish placebo effect from treatment effect. Therefore PACE was fatally flawed from the beginning. The rest of medicine has long accepted that unblinded studies don’t produce reliable information.
The problems with this study are numerous. In the media, the authors have reported information in a highly selective manner, so it falls to independent critics to inform the public.
The authors preferred treatments, CBT and GET, specifically attempt to change patient perception of symptoms while the other two interventions do not, and self reported outcomes are influenced by perception. Indeed the authors consistently made choices that would bias the results towards CBT and GET. It is therefore likely that CBT and GET would produce a slightly larger placebo effect. The small advantage of CBT and GET over the other two treatments, standard care and adaptive pacing, can easily be explained by this.
Reliable evidence of improvement is nowhere to be seen. The more objective outcomes such as walking test distance, fitness as measured in a step test, employment status, and so on, do not support claims that any of the tested interventions are effective treatments. Likewise, the follow up at 2.5 years shows no difference between the treatment groups.
The authors have resisted independent analysis of the data, sharing data only under conditions where they could control the outcome and conclusions. Data was released to one skeptic only after long legal battle which cost Queen Mary University London approximately £250k in legal fees. Is this the behaviour of researchers that are confident in their work?
I cannot list all the problems here. See David Tuller’s article series for a more comprehensive list.
PACE is a reminder that science needs greater openness and transparency because researchers are not immune to self delusion.
Dave, all of this has been covered extensively in the trial FAQs here http://www.wolfson.qmul.ac.uk/current-projects/pace-trial/#faq. Nothing dodgy and nothing to hide. It is perfectly acceptable to make changes to your analytic plan or indeed protocols during the conduct of a trial so long as you explain why, get TSC/DMEC approval, document it fully and make it clear in the publication. All of that was done. The 2012 Lancet paper doesn’t deal with recovery anyway, that was covered in a secondary later publication. In essence though they decided they were using a overly harsh set of criteria that didn’t match what most people would consider recovery and were incongruent with previous work so they changed their minds – before a single piece of data had been looked at of course. Nothing at all wrong in that- happens in vast numbers of trials. The problem arises, as studies have shown, when these chnaged are not properly reported. PACE reported them properly. And indeed I happen to think the changes were right – the criteria they settled on gave results much more congruent with previous studies and indeed routine outcome measure studies of which there are many. And re analysis proves the wisdom of that to be honest. But even then, using criteria that were indeed incongruent with previous work and clinical routine outcome studies, the overall pattern remains the same. CBT GET superior to pacing SMC. Most people who work in the field agree with them by the way – the criteria in the recovery paper itself (not the main lancet paper which never dealt with ) approximate with all the previous work – reasonable and meaningful but not the new penicillin. Am not going to respond to more comments on the same lines except to say it’s very likely that this has been addressed before and covered in previous responses by the authors and the trial FAQs. If you want to use the original recovery criteria you can, but I think that the ones that the trialists finally decided upon are more realistic to patients and clinicians alike . And finally it’s not a surprise that if you use extremely harsh criteria you get lower frequencies- no one has pointed out that the original criteria did indeed give recovery rates for Pacing and SMC that are also pretty daft and don’t match patient experience of either. People do get better on pacing and even just TLC- it’s just that more get better with GET and CBT, but still not enough. OK, enough already. Back to day job.
Simon Wessely wrote:
——————————————————-
In essence though they decided they were using a overly harsh set of criteria that didn’t match what most people would consider recovery and were incongruent with previous work so they changed their minds
——————————————————-
This is an absurdly misleading statement, and a good demonstration that the authors and their allies are an unreliable source of information.
The average age of PACE trial participants was about 39 years. People of this age should have a PF-36 physical function scores of about 93 (with 100 being completely healthy).
To enter the trial, participants needed to have a score of 65 or lower. The authors considered this sufficiently disabled to enter the trial.
While the trial was under way, the authors decided that a “normal range” of physical function was 60 or above. This corresponds to a person 80 years old, and is lower than the entry criteria.
Sources for this information can be found here: http://me-pedia.org/wiki/PACE_trial
I don’t have the energy to address all the misleading statements by Wessely.
With all due respect, Dr. Wesseley, it does still strike me as dodgy. If the PACE authors had simply told the press and the scientific community while publicizing their findings that their results showed *no* statistically significant improvement under their original protocol, I think we all would have had a much better sense of what this study meant: their interventions could lead to “recovery” only in the very limited sense of people who still could not drive, have jobs, exercise, or otherwise function with their age cohort… but not a “recovery” as a normal doctor or patient would define it, or even as the authors themselves had defined it years before. Instead, the authors spent more than a million pounds of university funds to avoid having to answer that question, for the obvious reason.
It also seems disingenuous to say that the authors couldn’t have at least suspected how their changes would affect the results. Rebecca Goldin, director of Stats.org, puts it better than I could: “You would not need to see the data to know that more patients would meet recovery criteria under the new, lowered bar. It’s like going to a middle school and defining “tall” to mean above 5’9”, and then before checking how many people qualify as “tall,” you change your criterion to 5’5”. You don’t need to know the heights of the people at the school to know that there will be more “tall” people. It’s definitional.”
In his post, Simon Wessely defends the altered “recovery” criteria used by the study authors.
He is defending – without explanation – criteria that include a physical-function recovery criterion of 60/100 on the SF-36 scale, which is BELOW THE LEVEL OF TRIAL ENTRY.
That is, patients could become more disabled during the trial and yet be classed as “recovered” by this criterion.
The same is true for the fatigue-scale recovery criterion.
Both thresholds were clearly labelled as post-hoc in the Lancet paper and so must have been post-hoc when they were applied in the later “recovery” paper in Psychological Medicine.
Simon Wessely writes: says “Am not going to respond to more comments on the same lines except to say it’s very likely that this has been addressed before and covered in previous responses by the authors and the trial FAQs.”
No, it hasn’t.
Then he says: “OK, enough already. Back to day job.”
So once again, a defender of the PACE trial ignores this screaming flaw, claims that it has been addressed when it hasn’t, and leaves the discussion.
Psychological Medicine should retract these analyses. Their existence in the literature is shameful.
As a graduated statistician i must say your interpretation is highly remarkable. For example: if these data were used in real medicine, the drug wouldn’t be approved by any scienetific commission.
Simon,
1. Nobody has claimed that The Lancet paper wasn’t CONSORT compliant, so I’m not sure why you’ve brought it up. No-one is saying that the changes weren’t made ‘by the book’. The criticism that has been made of the PACE authors is the reasons they gave for deviating from the pre-trial protocol were pretty flimsy – and they now seem somewhat spurious given the dramatic impact that the changes had on both the improvement and recovery statistics. The PACE authors have been unwilling to engage with their critics on this point – I know you weren’t an author but can you clearly explain why the changes to the protocol were necessary?
2. The main findings were that about 60% of patients improved and 20% of patients ‘recovered’ (whatever that meant) with CBT and GET, and both of these findings have been disproven by the independent reanalysis. The effects of CBT and GET are not ‘modest’ as you claim – they are either a) virtually non-existent or b) actually non-existent.
3. Consequently PACE has nothing useful to say about the causes *or* management of CFS/ME. It’s great that you – and “all the serious clinicians and academics” – recognise that this is a “genuine condition that causes much misery to patients, friends and family”, but frankly that just makes it all the more inexplicable that you continue to defend such a shoddy piece of scientific research which has perpetuated the use of ineffective treatments and consequently extended that misery. Patients have been telling you and the PACE authors for years that these treatments don’t work, and even now when the figures clearly show that they were telling the truth, you don’t seem to be willing to listen.
4. You’re right to say that we need more effective treatments – and more progress would have been made in developing these treatments over the last five years if time, energy and money hadn’t been wasted on developing ineffectual CBT and GET programs. If the PACE authors had made the trial data available back in 2011 – as they were asked to – we would have seen straight away that they got a null result and moved on to more promising avenues of research. I accept your point about the NICE guidelines predating PACE – but what you seem to be missing is that the independent reanalysis shows that the PACE trial (which as you point out was much larger) has actually disproven the results of the earlier studies into the effectiveness of CBT and GET – they clearly don’t work, as the PACE authors’ own two and a half year follow-up paper also demonstrated. I wonder if your confidence that PACE won’t be retracted might be somewhat misplaced.
Thank you also for acknowledging your role in developing CBT as a ‘management approach for CFS’. I appreciate that it must be extremely difficult to see a substantial part of your life’s work utterly disproven and invalidated, but you’re honestly not doing yourself any favours by continuing to defend bad science and ineffective treatments. The PACE authors produced a deeply flawed study and then spent five years trying to prevent anyone from seeing what they’d done. As you’ve pointed out above, you weren’t a PACE author – but the more you try to defend them, the more you’re going to be dragged into this sorry mess and tarred by association. I can appreciate the urge to defend your previous work, but you really are better off just accepting that CBT is an ineffective treatment for ME/CFS and leaving the PACE team to deal with the fallout.
Professor Sir Simon Wessel said:
“In my country (UK) there was already sufficient evidence to recommend both therapies in our NICE guidelines before PACE was published. PACE will not be retracted because there is no reason to do so, it is fully CONSORT compliant and so on, but if it had not happened the guidelines would remain the same.”
No Professor Wessely, there wasn’t sufficient evidence for the NICE guidelines, as you must know only too well.
In correspondence between Professor Hooper and Frances Rawle, Head of Corporate Governance and Policy at the MRC, Rawle unambiguously admitted that prior to PACE there was a lack of sufficiently strong evidence to inform the use of CBT/GET for ME/CFS:
http://margaretwilliams.me/2011/hooper-reply-to-mrc-rawle_26jan2011.pdf
Prof. Hooper said to Frances Rawle:
“You attempt to justify the MRC’s funding of the PACE Trial by stating:
“there was a lack of high quality evidence to inform treatment of CFS/ME and in particular on the need to evaluate treatments that were already in use and for which there was insufficiently strong evidence from random controlled trials of their effectiveness”.
That is a remarkable admission, since the NICE Clinical Guideline 53 of August 2007 relies upon the pre-PACE Trial “evidence -base” to recommend the use of CBT and GET nationally as the intervention of choice for ME/CFS, yet you state in your letter that there was insufficient evidence for the implementation of this nationwide programme of CBT and GET recommended by NICE in its Clinical Guideline 53.
In other words, on the one hand Professor Peter White was strongly promoting CBT/GET in his submissions to NICE because he asserted that there was sufficient evidence of their efficacy for their implementation across the nation, yet on the other hand he has received millions of pounds of tax payers’ money to carry out the PACE Trial because there was NOT sufficient evidence of the efficacy of the same interventions.
This can only mean that since August 2007 NICE has been promoting interventions and subjecting sick people throughout the nation to a regime for which insufficient evidence exists, a situation that raises yet more legal issues and ramifications, since the correct option for NICE pending the outcome of the PACE Trial was to have recommended the use of CBT and GET “only in research”, not to have issued recommendations for widespread clinical use when evidence of efficacy for those interventions was insufficient at the time the Guideline was published.
This raises the issue of exactly why the Guideline Development Group was so determined to implement nationwide CBT and GET on an insufficient evidence-base.”
Also in the PACE Trial Protocol, March 2007, White et al used the lack of evidence for the efficacy of CBT/GET as a justification for the PACE trial:
http://bmcneurol.biomedcentral.com/articles/10.1186/1471-2377-7-6
“Chronic fatigue syndrome (CFS, also called myalgic encephalomyelitis/encephalopathy or ME) is a debilitating condition with no known cause or cure. Improvement may occur with medical care and additional therapies of pacing, cognitive behavioural therapy and graded exercise therapy. The latter two therapies have been found to be efficacious in small trials, but patient organisations’ surveys have reported adverse effects.”
“Two independent systematic reviews have found that rehabilitative cognitive behaviour therapy (CBT) and GET were the most promising treatments for CFS/ME in secondary care [5, 10, 11, 12]. The published trials of these treatments were however also criticized for being too small, too selective, and for using different outcome measures. No other treatments for CFS/ME have so far been shown to be helpful in more than one RCT [5, 12].”
And from the full PACE trial paper:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3065633/#bib4
“The UK National Institute for Health and Clinical Excellence (NICE) recommend cognitive behaviour therapy (CBT) and graded exercise therapy (GET).2 Although this recommendation was supported by systematic reviews,4–7 supporting evidence remains restricted to small trials.4–7 Surveys by patients’ organisations in the UK have reported that CBT and GET are sometimes harmful, and have recommended pacing and specialist health care.8,9”
So we have both Frances Rawle, MRC and the PACE authors admitting that prior to PACE there was insufficient scientific evidence for the efficacy and safety of CBT/GET.
And here’s Dr Neil Abbott of MERUK’s clear and concise article exposing the flaws and limitations of the NICE evidence base:
http://www.meresearch.org.uk/wp-content/uploads/2012/11/NICE_Evidence.pdf
You’ve got to love the psychiatrists. After promising “cold facts”, Wessely introduces none to the argument. His comment is just obfuscation, weasel words and misrepresentation. He promises a lot but can’t deliver.
Which, when you think about it, is an apt metaphor for the treatment he endorses.
As someone who became deathly ill at age TEN, who went to 8 different doctors but was never sufficiently tested, then disregarded as having CFS, I know what it’s like to be told it’s all in my head. I’m 33 years old now, and have been to a number of head-shrinkers, been put on modified diets, gone on dozens of exercise protocols; all of which only made me worse. The study in the Lancet was quoted to me by many doctors as all I needed to feel better, but no matter which configuration of exercise and “mind over matter” denial that I was sick, have I ever gotten one bit better. When my health levels were monitored by a social worker, who was looking into my large number of absent days from school, she noted that my health actually declined when I followed the doctors schedule, instead of taking rest periods and listening to my body when it said I needed rest. I have faced not just doctors, but school officials, HR officials for work, friends and my own flesh & blood who have said not only do they not believe I’m sick but that it’s all in my head. Serious investigation and oversight must be put into the causes of CFS and real treatments be put forth which include the doctor actually believing the patient when they say that their is something wrong with their health. I do not believe that the study presented in the Lancet tells the whole story about the recovery rates the people in the study had nor does it show the damages inflicted to those who participated, i. e. did they need more rest time or medical attention after doing the exercises in the study. More information must be put forth for a full and complete understanding of its findings. If the names of the people involved are the concern of the studies leaders, then block out their names but let the rest of the information be released.
Dear Simon,
You’re not spoiling the party in any way.
When you say: “And indeed I happen to think the changes were right – the criteria they settled on gave results much more congruent with previous studies”, reckognising that changing criteria to fit the results you expect is OK, you’re just admitting this whole trial has nothing to do with science.
Thanks a lot
Dear Sir Simon,
I suggest you read the abstract for the PACE long term followup:
“There was little evidence of differences in outcomes between the randomised treatment groups at long-term follow-up.” [1]
As you have said, the PACE trial is indeed “a thing of beauty”. Patients are wielding it like Thor’s Hammer to smash your mountain of psychobabble.
It is certainly no mystery as to why QMUL spent 250,000 pounds in a vain attempt to hide the data. My crystal ball predicts a careful analysis of the data will reveal many patients deteriorated over the course of the study, and that the PACE fraud will result in numerous lawsuits.
Better save a few shillings from your day job to pay for lawyers…
[1] http://www.ncbi.nlm.nih.gov/pubmed/26521770
Professor Wessely, you must know that your and your colleagues’ work with its ‘modest’ effectiveness has for decades been used by governments as an excuse to withhold adequate funding from the search for more effective treatments for all those tens of thousands of us for whom your treatments are entirely useless. This doesn’t seem to bother you. I know you’re not trying to prove that ME is psychiatric so that you can prove ME ‘isn’t real’ but rather trying to prove that ME is psychiatric so that you can prove that psychiatric diseases are ‘real’. But you’ve failed to get your message across and you’ve used millions of us as cannon fodder in the process.
Incidentally, can you point me to any doctors who also have M.E. who have pronounced your treatments effective?
C’mon. The jig is up and you know it.
I agree with Anton Mayer and many others here, any study which denominates the *worsening* of disease in very disabled patients as “recovery” or “cure,” is a “sham” as Columbia U. Medical School’s Prof. Racaniello has written (on the Virology Blog). This is not even considering the paper’s other fatal flaws.
I think it is interesting that Wessely claims that the changes to the recovery protocol were approved by the DMEC as this has not been claimed by White in the past. The claims around approval changes with the DMEC are for the statistical analysis plan which does not mention recovery and as far as I can see does not mention protocol changes either.
I think there is a critical question as to who approved the changes to the recovery secondary outcome and using what information because the reasons for the changes given in the recovery paper are simply wrong.
Is Sir Simon, with his snarky opening comment, unaware how many of us that care about these things are not only unable to party, but unable to wok, unable to have a family and in my case unable to leave my bed?
Does he not accept that the CBT PACE trial manual lays out very clearly what he, Chalder and Sharpe have been telling the medical profession, the media and insurance companies for years and we patients feel keenly the stark lack of interest in biomedical research which is an effect of them being told we have a disorder we can reverse by changing our unhelpful beliefs and doing more.
How convenient for a dr who’s misrepresented the illness so drs think it’s remedied by behavioural changes to then use, in defence of the continued use and study of the treatments, that they’re the best we have. And how obvious it is to anyone not duped into dismissing the ME community, that there strong vested interest and face saving going on here in certain sections of the scientific and medical community. Never mind if it’s at patients expense. How clear it is to me that I lost 20 years of my prime years because medical establishment got it wrong and am loosing yet more precious time because they’re too proud to admit it.
Thank goodness this fraud is exposed at last. Changing protocols mid-experiment–and the authors’ refusal to release the results under the original protocols–should have been a glaring red flag to any doctor or journalist, much less the Lancet’s peer reviewers. Perhaps there will prove to be a modest benefit to PACE for some people; CFS is highly variable and it’s not impossible. But it helps nobody when researchers exaggerate their data to this degree.
As a fellow ME/CFS sufferer, I know you already know what this means to all of us out here thinking there must be something *really* wrong with us because the therapy worked for “the majority” but made things worse for us.
Thank you, from the bottom of my exhausted heart.
Makes me uncomfortable that we have lots of marketed pharmaceutical products purportedly for treating specific diseases out there when the diagnosis, and mis-diagnosis, of so many medical conditions seems to be at such a rudimentary level of understanding.
Yes, much more of the good science please, as free from politics as it can be. (Thanks Craig). And relentless exposure and punishment of the scammers, particularly when they are researching in the human health arena.
Thank-you Julie for putting the pieces of this story together and doing so succinctly. I too have witnessed the falsification of science, but when it invades the clinical arena, it is of more immediate concern. Congratulations to Rodale Press for publishing your book which I anxiously await reading.
I too, eagerly await Julie’s book.
Thank you!!! I, too, knew that this study didn’t make sense, but not before it made me feel terribly depressed that it was, once again, all my own fault (even though I’d been working at increasing my activity level and having therapy for 13 years+ now). You’re so right that sometimes the same activity I successfully did before will, on a different day, knock me out of commission for days. In the first 10 years small, identifiable actions (once it was even a fall in the yard) could set me back in bed for _months_. I can see my own improvement, but being a _reliable_ contributor to anything is not yet possible. I still hope. My hope is in the gradual work of finding biomarkers that will illuminate paths to treatment. Thankfully, breakthroughs are emerging. Once diagnoses are made clear and subsets identified, types of real treatments _will_ begin to emerge. Thank God for progress! I’m very grateful, too, for your work and that of others’ who are doing what the peer reviewers should be able to do for their illustrious magazines.
Good science is good and helps many. Those responsible get accolades. Bad science is bad and causes grief to many. Those responsible should be publically shamed. Even those who condoned by peer review should be shamed. Thanks to those who understand how much CFS actually does impact on people’s lives and it’s dismal research like that endorsed by Lancet that exacerbates the struggle to resolve.
Please. The PACE study was clearly flawed, that’s obvious. Anything from Queen Mary University should be given extra scrutiny, and the Lancet is not the journal it once was. But this article, with its ad hominem criticisms and failure to name the popular media as by far the worst culprit in the debacle, doesn’t help at all. Advocacy is not science and never will be. It’s politics.
It may take decades more to figure out what exactly CFS is, or even if it’s a single illness rather than a set of similar ones. Personal anecdotes are not science either; they’re irrelevant. At best they are indications that the research only addresses some of the affected people or some of the symptoms. If people would wait for the science to shake out over time rather than listening to science writers we might get some progress. I understand that you’re suffering and impatient, but all the screaming & rending of garments in the world is not going to help.
Except that science is very clearly also politics. Or at least, profoundly human. It was in this case. And in so many others (drugs that have been either put on or denied access to the market for reasons that have little to do with science, the lie perpetrated for 50 years that fat, rather than sugar, causes heart disease, the recommendations on thyroid and prostate cancer, now repealed, that lead to hundreds of thousands of needless surgeries in spite of the evidence). Advocacy, when it asks simply for good science, is on the side of science.
We have been waiting for science for more than thirty years to “shake it out.” In the US, we get only $5 million per year in research funding. That is also politics and we will never discover anything with that paltry amount of money.
Yes, the UK media have played a terrible part in this as well – BUT it was the authors who were holding press conferences claiming “recovery” and “back to their lives” to promote their research. That’s not science, either. That’s PR. I have never seen serious scientists in the US do this, or at least, not to this degree. You publish your result and then you wait for others to join the conversation. They were not tentative, they were not humble. They were self-sure and deeply irresponsible. Thousands of people are more sick, more disabled than they otherwise would have been, myself included. I don’t believe they will ever know or feel the weight of that, so deep is the self-deception, but that doesn’t make it any less true.
It’s a fascinating and tragic story and I am sure if you dug into the arguments and tried to understand the lived experience of this disease (which provides a zillion clues as to the possible cause and fundamental nature of the disease) you would think differently.
I just saw this article on a new study called “Metabolic Features of Chronic Fatigue Syndrome” published in the latest Proceedings of the National Academy of Sciences. Even though a naturopathic doctor is listed as a co-author, I hope it may hold some hope.
Retractions are rare? Check out retractionwatch.com