Hospitals, medical schools, businesses, and even the National Institutes of Health are routinely violating a federal law requiring clinical trials to be reported to the public, a STAT investigation recently found. In response, we asked experts to offer their thoughts on why low rates of clinical trial reporting is a problem, and what can be done about it.
Christine Laine: ClinicalTrials.gov registration has important shortcomings
Tim Errington: Science works best when it is open
Karl Schwartz: Patients need accurate clinical trial information
By Christine Laine: The timely and transparent reporting of clinical trials is of utmost importance. It has been furthered by organizations such as the International Committee of Medical Journal Editors. For example, the Committee’s trial registration policy markedly increased the number of clinical trials with prospectively documented methods in a public registry. However, the STAT investigation’s laser focus on whether the results appeared in ClinicalTrials.gov as an indicator of responsible trial reporting is troubling to me as a journal editor who oversees the peer review and publication of many clinical trials.
While the registration of results in ClinicalTrials.gov is a baby step toward responsible trial reporting, it has important shortcomings. Anyone who examines the registry will quickly see that results reported in this manner are difficult to digest and are unlikely to be meaningfully useful to patients and physicians as they make real world clinical decisions. Further, the registry allows investigators to post results that have not been vetted by peer review. In my more than 15 years as a journal editor, I have never seen a clinical trial report published without requiring substantive changes in response to expert peer review. Thus, reporting clinical trial results in journals after peer review, and in a format that provides context, detailed methods, and explanation of trial strengths and limitations, may be the most transparent, responsible, and useful way to disseminate what we learn from trials.
The STAT investigation does not appear to have gathered information about whether the results of the trials it reviewed had been published in a peer reviewed venue. Of course, if a serious effort to publish trial results remains unsuccessful, trialists should register their results. Anyone attempting to use results that have not been published in a peer-reviewed journal in their clinical practice or research should recognize that inability to find a home in such a journal could signal serious problems with the design, conduct, or reporting of the trial.
The world is quickly moving beyond registration of trial methods and results. There is a movement to require that clinical trialists share de-identified, patient-level data with others who wish to use these data in their own work. Such sharing holds great potential to advance science in a more efficient manner than ever before and will make registration of results passé.
Christine Laine, MD, is editor in chief of Annals of Internal Medicine and senior vice president of the American College of Physicians.
By Tim Errington: Science works best when it is done in the most open way possible. Unfortunately, open science is something we mostly talk about today instead of something we practice.
Failure to accurately, completely, and quickly report the results of clinical trials, as well as unexpected findings and harmful side effects, poses hazards for researchers, clinicians, and patients. Not reporting this information leaves gaps that make it difficult to fully determine the effectiveness of a treatment or to understand the risks of trying a new medication.
Reporting clinical trial results can feel like a lot of work. It’s usually done in a separate process after the data have been collected, analyzed, and reported in a journal. That’s no excuse for delaying or failing to report results, but it is a barrier. In addition, there are no incentives for sending the required data to ClinicalTrials.gov, nor are penalties enforced for failing to deliver this information. This combination leads to the dismal results seen in the STAT investigation on clinical trial result reporting.
It’s time to shift the incentive structure, which currently focuses on publishing the results of clinical trials — especially positive results — rather than on the quality of the research that has been done. It’s also time to make it easier for researchers to work in more open and more streamlined ways.
The Center for Open Science and others are working to do just that. Researchers have traditionally generated ideas, collected data, run analyses, and disseminated results in virtually separate processes, and they often have to regenerate information for each step. We now have the technology to unify these processes, and should start to use it more widely.
It’s also important to link related material such as registration information, data collected during the trial, analysis methods, publications based on the data set, data that didn’t make it into the publication, related analyses, and the like. That would let individuals see all of the information about a trial in one place. Open Trials is working to achieve such linkage.
Although STAT’s focus on reporting to ClinicalTrials.gov was much needed, the concerns go beyond this registry and database. We must extend this kind of reporting around the world.
The more open the scientific process and the more information we can share, the more the global community will benefit.
Tim Errington, PhD, is project manager for Meta-science at the Center for Open Science in Charlottesville, Va.
Karl Schwartz: Thank you to STAT for its investigation into the disappointing failure to report clinical trial results as directed by the Food and Drug Administration Amendments Act of 2007.
The STAT report identifies failures by eminent institutions on the forefront of clinical research. It also identified some of the underlying causes, which is an important first step to help these institutions to rise to the challenge.
Reporting research results is a critical aspect of collaboration that informs decision making by researchers, physicians, and patients. It also guides the direction of future research.
Timely and full reporting shows respect to clinical trial participants who, in volunteering, have shouldered potential risks. Selective reporting can potentially lead to inappropriate medication use. For example, when negative trials are not published, a drug’s benefit can be exaggerated. This could lead to inappropriate off-label use, which increases the expense of treatment and exposes more patients to possible harm.
The STAT investigation is helping shine a light on an issue that is very important for patients. Going forward, it would be important for the federal government or an independent organization to formally evaluate on an ongoing basis how well institutions report clinical trial results.
Compliance with clinical trial reporting requirements is vitally important. It will foster patient and public trust in the research system, while improving its quality and efficiency.
Karl Schwartz is the cofounder and president of Patients Against Lymphoma, a nonprofit advocacy organization that provides support, education, and evidence-based resources on lymphoma and its treatments.
