Gut Check looks at health claims made by studies, newsmakers, or conventional wisdom. We ask: Should you believe this?

The claim:

The large clinical trials needed for federal approval of new cancer drugs often way overstate how effective the treatments will be in the real world, two cancer physicians argued recently in JAMA Oncology.

Tell me more:

The point of cancer clinical trials — in which some volunteers take the experimental drug, others receive standard care with existing drugs, and the groups are compared on measures such as whether their tumors shrink, how long they go before tumors return, and how long they survive — is to see whether a drug is safe and effective enough to be sold to American patients. The whole process rests on the premise that the trials give an accurate indication of safety and efficacy among cancer patients in general, not only those who are eligible for and selected for the trial.

Trouble is, participants in clinical trials are unlike the overall cancer population, oncologists Dr. Sham Mailankody of Memorial Sloan Kettering Cancer Center and Dr. Vinay Prasad of Oregon Health & Science University pointed out in JAMA Oncology. They’re younger, healthier, better plugged in to the health care system (or they wouldn’t have found a trial in the first place), better educated, and wealthier (it’s often necessary to travel multiple times to where the trial is being conducted).

article continues after advertisement

One example: 60 percent of US cancer patients are older than 65 and 31 percent are older than 75. But in clinical trials of cancer drugs from 2007 to 2010, when the FDA approved 24 new cancer drugs, 33 percent of participants were older than 65 and 10 percent were older than 75, researchers reported in 2012.

In addition to being younger, whiter, and healthier than other cancer patients, those in clinical trials tend to be “highly motivated and to have a strong support network,” both of which can improve survival, said Dr. Hanna Sanoff of the University of North Carolina Lineberger Comprehensive Cancer Center. The medical care in clinical trials, which is often conducted at academic medical centers, is also arguably better than in the real world. All those factors make cancer patients more likely to do better in a clinical trial no matter what treatment they’re on.


Concerns about a gap between clinical trials and the real world are not just theoretical. In a 2016 study, Sanoff and her colleagues compared results from the pivotal clinical trial of the cancer drug sorafenib, which led the FDA to approve it in 2007 for advanced or metastatic liver cancer, to what’s happened since.

In the trial, patients on sorafenib lived a median 10.7 months. The comparison group lived a median 7.9 months. While an extra 2.8 months of life isn’t exactly a home run, it’s not unusual for such molecularly targeted cancer drugs. What was unusual was that the 2.8-month benefit was relative to patients receiving a placebo (because there were no drugs that were effective against hepatocellular carcinoma), a comparison that should have put sorafenib in the best possible light.

Sign up for our Cancer Briefing newsletter

Please enter a valid email address.

When Sanoff and her colleagues examined a cancer database run by the National Institutes of Health to see how people with liver cancer fared on sorafenib in real-world use, the results were sobering. The roughly 400 patients who received sorafenib were sicker and five years older (70 vs. 65), on average, than those in the clinical trial. And their median survival was three months — well short of the 10.7 months in the trial. In comparable patients who did not get sorafenib, but only supportive care, median survival was also three months. So there was no benefit from taking the drug.

Real-world patients are so different from those in a clinical trial “as to transform a cancer drug with marginal effect into an ineffective drug,” Mailankody and Prasad wrote. “The case of sorafenib is not isolated.”

The verdict:

If cancer patients are similar in age, presence of other illnesses, socioeconomic status, and other characteristics to those in a clinical trial, they might do as well, but for everyone else, the trial results probably overpromise.

Sign up for Morning Rounds

A daily dose of health and medicine news — and a finalist for Digiday’s best email newsletter.

Recommended Stories