TORONTO — The rejection seemed like a bad joke.
Developmental biologist John Cobb had applied for a Canadian government grant to research a gene that causes malformed limbs. The anonymous reviewer looking over his proposal acknowledged the gene contributes to serious growth disorders, but added: “Be that as it may, short stature is a cosmetic problem, not a disease.”
“I laughed,” Cobb said. “It was a sad laugh.”
It’s been an ugly summer for science in Canada.
The Canadian Institutes of Health Research, which funds most medical research in the country, recently rolled out a wildly unpopular new system for assessing grant proposals — and the first rounds of decisions have sparked widespread outrage, including an open protest letter signed by more than 1,300 researchers.
Health Minister Jane Philpott recently ordered CIHR to hold an emergency meeting with researchers to fix things, and since the gathering last month the agency has taken some steps in that direction.
But many researchers feel the damage has been done. Two grant cycles were cancelled while the new system was implemented, which led to a huge backlog of funding requests this spring. Yet under the new system, 87 percent of applicants in this cycle were rejected — a record high that puts countless careers in doubt.
Women scientists fared especially poorly. So did young researchers and those focused on basic science.
“Labs have certainly closed down,” said Jim Woodgett, director of research at the Lunenfeld-Tanenbaum Research Institute at Mount Sinai Hospital in Toronto. “There’s been a large decrease in trainees, a decrease in post doctoral fellows, and phenomenal technicians have had to be let go.”
Cobb, whose previous five-year grant from CIHR expired in 2014, is now operating by deficit spending in his lab at the University of Calgary. He has three graduate students he’ll either have to let go or stop paying.
He will also have to kill half his mice, which were specially bred — with CIHR funds, he notes — to carry human versions of the gene that causes malformed limbs. Keeping them is just too expensive.
The Canadian research agency “is trying to reform everything at once, and while the budgets have been shrinking,” Cobb said. “It’s like trying to fix an airplane while it’s in flight — it’s disastrous.”
#pscream Fave reviewer comment; "short stature is a cosmetic problem not a disease"; therefore limb deformities we study somehow not worthy?
— John Cobb (@JCobbinCalgary) July 16, 2016
CIHR President Alain Beaudet, who has faced calls to step down for ushering in the funding reforms, was unavailable for comment. But he issued a statement, posted on the agency’s website, acknowledging scientists’ concerns and saying they “must be addressed since CIHR can only be successful if it has the support and confidence of the research community.”
‘Whole hog into a massive change’
Competition for research grants is always fierce. But the new system, created in part with efficiency and cost-savings in mind, injected a fresh level of uncertainty. “We’ve jumped whole hog into a massive change,” Jane Aubin, CIHR’s chief scientific officer, said last year in a webcast presentation at the US National Institutes of Health.
Among those changes: Using a computer algorithm to assign reviewers to grant proposals based on keywords. Problem is, the algorithm often got it wrong, pairing reviewers with grants in subjects they actually didn’t understand.
“I had grants where reviewers had to openly declare, ‘I have no expertise in this area,’” said Lisa Porter, a research scientist at the University of Windsor who chaired one of the virtual panels in the new system.
Scientists were appalled: “You can’t replace experts with years of experience in the field with a dumb-ass algorithm looking for keywords,” said Rod Bremner, a University of Toronto professor and senior investigator at Mount Sinai Hospital’s Lunenfeld-Tanenbaum Research Institute.
When a computer glitch accidentally (and briefly) revealed the identities of reviewers, some applicants learned their grant requests had been critiqued by early-career postdocs who usually wouldn’t be thought qualified to grade proposals from more senior researchers.
“CIHR was really scrambling to find people to conduct reviews,” said Woodgett, adding that with the spike in funding requests, the agency even asked a technology transfer officer and a librarian at McGill University to take part.
Perhaps the most contentious change was abolishing face-to-face peer-review panels in favor of an online comment period, in a bid to save money on flying scientists to Ottawa for the meetings. CIHR recently agreed to alter this policy, but in-person reviews will still only be granted to researchers who make it to a short list of top contenders. Some researchers feel that doesn’t go far enough.
“When you meet face to face, and you’re there with colleagues, you have to know your stuff,” Porter said. Letting reviewers submit comments online led to a far less rigorous process, she said. It also gave the scientists requesting funding no chance to answer questions or correct misconceptions.
Many researchers felt the results were disastrous.
Bremner, for instance, applied for a CIHR grant to investigate cone photoreceptor cells. Those are cells in the retina that give rise to a childhood eye cancer called retinoblastoma. Yet this appeared to be news to the person who reviewed and rejected Bremner’s proposal.
The reviewer’s comment: “project focused on the developmental biology of cone photoreceptors, and its relevance to retinoblastoma is unclear to me.”
“How can I study the origins of retinoblastoma without studying cones?” Bremner said. “You have to be a complete idiot not to see the relevance. If this was said at a face to face meeting, there would be absolute silence, and a lot of shuffling of feet … With this [new system] there’s no accountability. I can say something completely asinine and even write it down, and that’s it.”
“project focused on the developmental biology of cone photoreceptors & its relevance to retinoblastoma is unclear to me” #PScream rev of day
— Jim Woodgett (@jwoodgett) July 18, 2016
In other cases, it was the guidelines that reviewers didn’t seem to grasp. For example, the online grant application only permitted researchers to type a maximum of 1,750 characters to summarize their projects — or roughly 350 words.
Yet one reviewer, who had printed out the application form, docked the researcher for a skimpy description because it took up so little space.
It was, says Woodgett, “unbelievably boneheaded.”
“Peer review is always a brutal process,” Woodgett says, “The difference here is that people are not getting funded for reasons that make no sense.”
For years before the changes were implemented, many researchers, prominent scientists and university presidents all warned that they would undermine the quality of reviews and the $1 billion CIHR distributes annually in grants. But under the Conservative government of former Prime Minister Stephen Harper, which had a notoriously frosty relationship with scientists, appeals to the health minister “went nowhere,” Woodgett said.
Women at a disadvantage
Even if the system is fixed under the new, Liberal government of Prime Minister Justin Trudeau, the bungled rollout could have a long-term effect.
Younger researchers — many of them women — who missed out on grants this year, after the drought of the last two funding cycles, “won’t have the funding to get their labs up and running and be productive, which then again puts them at a disadvantage when they apply for funding in the next round,” Porter said.
Those who study basic science also took a beating. The success rate among researchers who study model organisms such as worms, fruit flies, and zebrafish — in which major discoveries have often been made — was a paltry 3.8 per cent.
Julie Claycomb, a molecular geneticist at the University of Toronto who has been tracking those results, said she’s heard that some reviewers simply didn’t have the expertise to understand the grant proposals. Others felt the work lacked any immediate application for patients.
Claycomb, who studies epigenetics in the worm, was told her proposed project “will not translate to human systems.”
While she’s hopeful that things will improve, Claycomb said speed is the key: “We need to move quickly, or we are going to lose some outstanding scientists.”
Indeed, an online survey trying to gauge the impact of CIHR’s funding changes has found that 25 percent of the 410 people who have responded so far are seriously considering moving away from Canada. And 70 per cent said they are delaying the hiring and mentoring of graduate students or post-docs.
“These decisions will have significant repercussions for years to come for health research in Canada,” said Liisa Galea, a neuroscientist at the University of British Columbia who’s leading the survey.
As for Cobb, who grew up in Tennessee and moved to Canada in 2006, the recent rejection has made him pessimistic about his chances of securing further funding.
“I definitely have thoughts about coming back [to Tennesse], I won’t deny that,” he said. “But I like Calgary. I’d like to stay.”