Worried about America’s biomedical research enterprise, academics have persistently sounded the alarm over slow growth in the National Institutes of Health budget. The Trump administration responded to those concerns in its 2018 proposed NIH budget by recommending a 22 percent budget cut and a reduction in support for indirect costs. While those cuts never came to pass, they marked a manifestation of political will to examine spending at the NIH.
There’s no question that the NIH could be a leaner and more innovative organization. But that would require elemental reforms in how it supports indirect costs (more on those in a minute) and its reliance on study sections to review grant proposals.
We propose two key reforms and offer a model that could reinvigorate innovation at the NIH while also funding “more for less.”
Reform support for indirect costs
Most NIH grant proposals seek funding for two types of costs. Direct costs include salaries, travel, equipment and supplies, and the like that directly support a project or activity. Indirect costs, also known as facilities and administrative costs, cover expenses such as buildings, maintenance, and administration.
At the NIH’s inception in 1887, scientific grants covered only direct costs. In the 1940s, recognizing that this model favored wealthy institutions, the NIH began to include indirect costs, initially capping them at 8 percent of the direct costs of the grant. Over the intervening decades, the cap increased to 26 percent and the NIH also created individualized indirect rates that now average 52 percent. With the NIH budget growth barely exceeding inflation, increasing indirect expenses directly inhibits the funding of new investigators and new grants.
In 2017, one-third of the more than $26 billion in grants the NIH awarded went to indirect costs. By comparison, the Gates Foundation reimburses indirect costs at 10 percent of the direct cost; the EU has a proposed flat rate of 25 percent starting in 2020.
The current system of indirect funding perpetuates systemic inequity favoring large, wealthy institutions. In 2017, the wealthiest 10 percent received 90 percent of NIH research funding. Recent research has identified large disparities in grant success rate and funding across a number of domains — including race, gender, and age — that severely limit the diversity of funded researchers and greatly hamper scientific progress. Increasing indirect costs in the setting of a flat overall NIH budget have perpetuated this inequity.
We propose that an institution’s indirect rate be placed into one of three tiers — 10 percent, 17.5 percent, and 25 percent — that are inversely tied to the institution’s rolling three-year average of NIH funding. Institutions with the highest record of NIH funding would receive the lowest indirect rate, and vice versa. By our conservative estimate, this change would release $2.2 billion that could be dedicated to funding creative research from new investigators who, lacking publication track records, could start their careers early instead of receiving their first grant at an average age of 42.
Reform the NIH study section
The process by which the NIH funds non-NIH investigators is largely directed by researchers from outside the NIH. Research areas are divided into hundreds of so-called study sections, each largely comprised of 10 to 30 academics. Each study section covers a narrowly defined area, such as cancer biomarkers and cellular mechanisms of aging.
Members are recruited by word of mouth, often through existing study section members. To serve on a study section, a researcher and must have an R01 grant, a requirement that excludes a large portion of the research community. Medical staff members at the NIH — some of the best clinician-researchers in the country — are not involved in grant review.
Study section members agree to three-year commitments, with oversight by a chair and an NIH staff member. Each grant submission is reviewed by at least two study section members, who then present the grant application to the entire section. After discussion, the study section members vote to assign the proposal an overall numerical score. The proposal’s score is compared to a funding cutoff score. Those meeting the cutoff are typically funded.
At 72 years old, the study section model for grant review is ripe for revision. Peer selection of study section members can result in groupthink. Personal animosity within the research community can result in the exclusion of individual researchers from grant funding or serving on a study section. With flat NIH funding and pressure to produce near-term measurable results, innovative ideas with limited supporting data promoted by new researchers are less likely to receive funding.
“Safe science” — mechanistic hypotheses with outcomes that are easy to predict and measure — performed by well-established researchers tends to be favored to demonstrate the sound use of federal funds, meaning that paradigm-shifting work is usually not funded. Yet established investigators with high NIH funding have decreased return on investment and productivity, as shown by research from the NIH’s own Office of Extramural Research.
Given this system, it is no surprise that our country faces a languishing pharmaceutical product pipeline in multiple critical product areas, a lack of meaningful innovation in service delivery, and other research. Like high-octane gasoline, the NIH extramural grant program is supposed to power basic medical research in the U.S. Yet many therapeutic areas are out of gas.
Research into new antibiotics is just one example. The U.S. and the world are running out of drugs to treat serious gram-negative infections, placing vulnerable groups such as cancer patients receiving chemotherapy at higher risk of complications, including death. The CDC estimates that in the U.S. alone, antibiotic resistance results in more than 2 million illnesses each year, along with 23,000 deaths, $20 billion in direct expenses, and more than $35 billion in lost productivity.
What was once a robust research space is now a desert: New drug applications for antimicrobials have declined from more than 30 new antibiotics approved in the 1980s to less than 10 new approvals in the first decade of the new millennium. Dr. Anthony Fauci, who directs the National Institute of Allergy and Infectious Diseases, has repeatedly highlighted the need for increased research in this area.
Look to the FDA review model
The Food and Drug Administration offers several models for review the NIH could look to for inspiration. The Center for Drug Evaluation and Research and the Center for Biologics Evaluation and Research serve as market-entry regulators for pharmaceutical products in the U.S. They employ thousands of expert staff members who serve as reviewers or as hybrid regulators, researchers, and clinicians.
FDA reviewers have helped nurture the creation of product spaces such as targeted oncology, approving 64 new products in 25 years, while fostering incremental innovation elsewhere, approving 1,222 new drugs since 1950. Product area advisory committees of outside experts supplement this community, serving as external sounding boards. We saw this system at work first-hand as a reviewer (B.J.M.) and intern (K.R.) for the Office of New Drugs at the Center for Drug Evaluation and Research.
The NIH would benefit from transforming from a volunteer to a professional review process for grant applications. Funded by reduced indirect costs, with staff members recruited from academia and industry, the creation of a professional staff would promote intellectual diversity and institutional memory, decrease bias towards individual investigators, and supercharge the funding of revolutionary ideas. Area-specific external advisory committees could assist in answering difficult questions.
Looking forward
The NIH extramural grant program is in desperate need of reform so it can embrace young investigators and those with paradigm-shifting ideas. Runaway indirect costs for large, wealthy institutions needlessly consume billions of dollars instead of directly supporting up-and-coming scientific investigators. Groupthink within the study section model paradoxically paralyzes revolutionary ideas. Pharmaceutical companies, lacking the fuel of a blossoming basic research environs, struggle to find molecular targets to attack disease in high need areas.
As we seek to reinvigorate biomedical research in the U.S., we must leave no stone unturned: It is time for change at the NIH.
Brian J. Miller, M.D., is an adjunct assistant professor at the University of North Carolina Kenan-Flagler School of Business and an internal medicine resident at MedStar Georgetown University Hospital. He previously served as a reviewer in the Office of New Drugs at the FDA’s Center for Drug Evaluation and Research. Kyle Richardville, M.D., is an internal medicine resident at the Cleveland Clinic Foundation. He previously served as an intern in the Office of New Drugs at the Center for Drug Evaluation and Research.
As someone who regularly reads reviewer comments from a broad number of study sections, it is very clear that no one teaches reviewers how to review. Many of the complaints listed about peer review can be traced to just this problem.
“Groupthink” was not apparent in anything I read, since that would mean that comments would be highly redundant between reviewers. That has not been the case in my experience. Rather one sometimes wonders if the reviewers were actually reading the same proposal!
The bias for mechanistic, easily achieved grants is also a by-product of poorly trained reviewers. Most reviewers approach a grant proposal the same way they would a manuscript. However, reviewing a completed project should not be the same as reviewing future work; the criteria should be radically different. However, despite NIH’s instructions to reviewers about the criteria to be used, most reviewers stay in their comfort zones and favor, unconsciously or consciously, projects that seem more complete, manuscript-like, than those blazing new intellectual territory.
Much of what is wrong with the peer review system could be easily solved with a feedback mechanism for reviewers. Reviewing seems to be learned more by trial and error–at the expense of submitters–than any concerted effort to keep the reviewing process consistent, fair and inclusive. Reviewers need help to become more effective, and feedback would start that process. From the faculty I work with, they take their reviewing seriously and are very aware of the impact they have on careers. However, many of them feel very isolated as they are left to deal with a pile of new proposals in a short period of time. They become hermits just to get through the workload, and no way of knowing if they are doing it well and in the best and easiest way.
Keep Peer Review: While voluntary peer review may need some tweaking, it should not be eliminated. Peer review was developed as a way to effectively judge the merits of proposals and manuscripts by individuals in closely related fields, particularly in the basic sciences, since concrete outcomes (drugs or devices) for those areas are not easily or immediately discerned. No one can predict which areas will be more fruitful or foundational with time. Science moves ahead by accumulation of knowledge, with no way of knowing which bits of knowledge will prove most valuable in the future. Those of us more senior are aware of new areas of research that opened up when work long thought out-of-date or at a dead-end suddenly made sense with new techniques. Who could have predicted it? Peer review of basic sciences, judging just the merit of the proposed work and not whether it is trendy or likely to yield a new product, keeps areas open when a more practical-minded process says close it. Professional reviewers are more likely to favor immediate return on investment at the cost of long-term accumulation of scientific knowledge without quantifiable value. That is a huge detriment to Science.
There are other considerations in peer review. When study sections regularly met face-to-face, it was a way to network with others who may not go to the same conferences but who might actually be able to expand your (and their) research horizons. Remote meetings do not do that, since most everyone is multi-tasking out-of-sight and only pay attention when it is their turn! I know, I have been in the room while someone was supposed to be in study section!
A better selection of true peers would help. Mixing clinicians and basic researchers has been a disaster and should not continue. They do not have the same mindset. I have seen and heard some very weird, and very unscientific, comments when researchers who generally work with animals look at human projects and vice versa. You cannot manipulate your subjects the same way! Science is not mix and match! Making sure everyone comes to the table with the same mindset is essential. In particular, professional reviewers cannot be true peers. No matter what their previous experience, once they quit doing research, their priorities shift, and as employees, they have to be responsive to their employer’s specifications, not the overall good of the scientific endeavor.
Peer review still has merit in forging broader networks of related researchers and in making sure science is well-done and well-thought out. Let’s make sure it is operating effectively, but do not disparage its historical nature.
Like I said in my initial comment, I do not agree with everything proposed in this article. However, I would prefer if the comment dialogue focused on the issues rather than on personal attacks, which come across as snobby/elitist/ageist and have a “get off my lawn” tone. The authors do not appear to have industry ties (from what I can tell), and “conflict of interest” type attacks could similarly be raised towards other commenters who may be benefiting from the status quo and lose their privilege in a reformed system?
My questions for those who know more than I do about these issues:
1. Is the current system where institutions negotiate overhead/F&A rates to levels that, on face, seem excessive and inequitable the best way to fund the highest number of investigators and research projects? Why are the rates so much higher than foundation grants? Do excessive overhead rates play a role in supporting the rapid growth of university administrators?
2. Could there be a better approach to recruiting members to NIH study sections that could lead to a better selection of research projects to fund?
3. Could there be a role for grant reviewers who are, themselves, not dependent on NIH grant funding and may have fewer conflicts of interest?
This discussion of indirect costs from MIT may be helpful: http://web.mit.edu/fnl/volume/295/zuber.html
Jose, answering your questions:
1. I agree with the authors that the current indirect cost system is unfair and ineffective. As one of our university administrator once put it, the indirect costs money goes to the top and trickles down to the bottom, the top being the high university offices and the bottom being the investigator. The book “The Fall of the Faculty” has some nice information on how the indirect cost system has been abused, event to the extent of corruption. As you point out, there is a takeover by university administrators, also documented in that book. The problem is that the indirect costs system became long time ago the main system by which the federal government supports universities. The fact that the money is tied to the grants ensures that universities with the stronger research programs get the most money, which I think is a great incentive for universities to support scientific research. If you tweak with this time-honored system you’ll have to watch out for unwanted consequences. For example, universities increasing even more tuition costs and losing interest in scientific research.
2. I don’t know what system NIH uses to recruit members of the Study Section. In my experience, this is one of the main function of the Scientific Research Officer (SRO) of the Study Section. These are professionals with excellent scientific training (most are ex-PIs themselves) that devote all their time to this job. They do not have a stake in who gets awarded grants and typically are very protective of their independence. They are also highly supervised by a system that is very aware of the potential for bias. I once recommended a colleague to the SRO of the Study Section, and he got it. However, SROs take this type of recommendations as just one more source of advice.
3. Contrary to what the authors of the article say, I know members of my Study Sections who are not, and have never been, PIs of a NIH grant. However, when you are both a PI of a grant and a grant reviewer it works well in both directions. First, because you have written grants yourself you know the type of pressures and problems that the grant writer faces and can read between the lines. This can work both in favor and against the grant writer. You have a fine-tuned bullshit detector, for one thing. Second, reading other people’s grant makes you a better grant writer. This is good motivation. Otherwise, why would you do the enormous amount of work involved in reviewing grants, for basically no pay? Every grant cycle (3 cycles a year) the NIH gets then of thousands of grant applications. Reviewing one R01 grant takes me 1-2 days. If you have to pay highly-trained scientists to review grants, this would involved a significant amount of money, that the NIH would have to divert from funding science. In fact, the system in place right now is a dual system in which both scientists outside the NIH and NIH-hired scientists (SROs and Program Officers) participate in the review system. To convert it in a system of only scientist-administrators would be a big step backward.
Such bullshot who knows where to begin. But the most glaring error is claiming somehow that FDA review “nurtures” innovation in driveway development. FDA is concerned first with ensuring that new treaents are safe, and second, that they’re effective. The FDA does not nieture innovation. They ensure that drugs do not harm patients, and that they have some efficacy. Why this system should work in promoting discovery in basic science is one of the many, many missing pieces in this story.
Any time one of these industry mouthpieces uses the words “Innovation” or “Lean,” it is time to wake up. The problem with all of our federal agencies, including the NIH is too much industry interference, and too many of these underfunding, and undercutting operations. Our government science is under attack, by profiteers. The FDA “Streamlining” truly terrifying, and has led to a lot of expensive American deaths.
As Steven Munger points out, the authors know very little about the Study Section system and make several factual errors. If they had participated in Study Sections they would know that the system actually works quite well and have some clever safeguards against bias and group thinking. I have participated in several Study Sections for about 8 years and have been funded by NIH and the VA for 18 years. Grants are scored by 3 reviewers (not 2!) who are blinded to the reviews of the others. Once they can see the other review’s comments and scores they cannot modify theirs until the grant is discussed in the Study Section. This ensures independent thinking. Discussing the grant in front of a Study Section formed by some of the best scientists in the field can have a great impact in the reputation of the reviewer, creating a motivation for quality reviews. Discussions of the merits of grant are sometimes quite lively. Scientists are trained for independent thinking and love to disagree with each other. “Innovation” is one of the five criteria for which a grant is scored, so this is always carefully examined. Moreover, there are grant mechanisms like the R21 and the CEBRA grants that geared towards innovation and require little or no preliminary data. Therefore, much of the criticism of these two young MDs in not based on reality. The system actually works quite well and is being continually tweaked to work better. What they propose is to take away grant review from the scientific community and hand it over to a bunch of bureaucrats, which would be a total disaster.
This article is a hot mess. It is filled with factual errors and a general ignorance about the system for extramural funding. Just one example…new study section members are not selected by other members, but by NIH staff who are required to balance not only expertise but factors such as the type of institution and region of the country the individual represents. Perhaps if the authors had ever participated in peer review they would also know that appointments to study section are typically 4 or 6 years, that grants typically get three reviews, and that grant scores are typically percentiled against other grants to help normalize scaling differences between study section meetings. There are a lot of things that could be done to improve NIH processes and help advance science in the US. This uninformed article unfortunately misses the boat.
I don’t necessarily agree with every point in this article, but I think the authors make an excellent point about the indirect cost issue. It seems ridiculous that many institutions with multi-billion dollar endowments have indirect cost/overhead rates that are over 50% of the grant amount (i.e., Harvard Medical School on-campus research = 69.5% indirect rate). They then use their influence and financial resources to lobby the government to maintain/increase these rates, and smaller institutions without such resources make do with much smaller overhead rates.
I think if people understood this issue better, they would be furious, because in a way, US taxpayers are subsidizing the endowments/capital of the most wealthy/elite institutions, which are inaccessible to the vast majority of people to attend.
Foundation grants typically place a cap on the overhead/F&A rate at a percentage far below the NIH negotiated rate. A fixed or limited number of tier rate plan would make grant funding much equitable to institutions across the country, reduce the influence of wealthy institutions in lobbying for excessive rates, allow for a significant increase in the number of grants funded.
Elite universities would obviously not like this sort of approach, but they could adjust by reducing their internal administration expenditures (as some universities such as Purdue have done) and using funds from their endowments to make a smooth transition to a more equitable system.
The NIH might well form DARPA-like sections within intra- and extra-mural arms. (High risk, high reward, small, agile etc)
The proposals for funding might also follow The Heilmeier Catechism (https://www.darpa.mil/work-with-us/heilmeier-catechism )
Grants might also be reviewed without revealing the identity of proposers.
Could we have some clarity on the qualifications of the authors please STAT? Have they served on study sections and competed successfully for NIH grants? (And if not, do they believe that this is because of “personal animosity”?)