As Covid-19 surges, researchers amass lung scans to build AI systems for patient care

At first, the images of lungs infected by the novel coronavirus were hard to come by. It was early in the pandemic, and Joseph Paul Cohen, a researcher at the University of Montreal, was trying to stockpile radiology scans to train an artificial intelligence model to recognize warning signs of severe illness.

With so few images available, the work was next to impossible. But in recent weeks, the resurgence of Covid-19 in the U.S. and other hotspots has solved that problem, allowing him to amass hundreds of lung scans from clinical reports published around the world. “We’re at a good number now,” said Cohen. “There’s a sufficient amount to start doing this.”

Little more than six months after the pandemic emerged, a number of researchers and companies are already testing the ability of AI systems to aid diagnosis of Covid-19 from lung images, and with the data on patients flowing more freely, researchers like Cohen say they can begin to build more reliable AI models that seek to predict the severity of disease, gauge patients’ response to various treatments, and determine whether they are likely to need a ventilator or transfer to an intensive care unit.

Radiologists said demand for such AI-powered tools may increase significantly in the months ahead, as surging numbers of infections threaten to strain medical resources in the United States and around the globe.

“If this hangs around, having (AI) to help triage cases could be valuable,” said Bibb Allen, chief medical officer for the data science institute at the American College of Radiology. “But we’ve got to build it first. That’s kind of where we are.”

The college, the professional society for radiologists, has created a repository of Covid-19 images from around the world and posted use cases on its website that may help in managing these patients. The organization does not recommend use of imaging for screening patients, due to high cost and the difficulty of distinguishing Covid-19 from other lower respiratory infections. But its use cases include efforts to use AI to characterize the extent of illness based on lung images, to help treat patients and manage hospital resources.

Researchers at academic hospitals across the United States are using CT scans, chest X-rays, and lung ultrasound images to help build predictive AI models to support treatment of Covid-19, including at Stanford University, Ohio State University, University of Pennsylvania, and Emory University.

Multiple AI teams have formed within the University of California. On its San Diego campus, a research team worked with Amazon Web Services to train an AI to spot early signs of pneumonia, a common cause of death in Covid-19 patients. The system, which analyzes chest X-rays, is not intended to diagnose Covid-19, but to flag patients that may become severely ill.

In one case, the AI detected signs of pneumonia in a man with no symptoms who was given a chest X-ray for other reasons. The patient was then tested for Covid-19 and found to be positive. He was admitted when symptoms developed and eventually recovered.

“We were much more proactive in hospitalizing him early,” said Christopher Longhurst, a physician and chief information officer at the University of California, San Diego. “We’ve seen a number of cases now where the AI is clearly influencing clinical management.”

Longhurst said the hospital is finishing up a clinical study of the use of the AI in treating Covid-19 patients and expects to publish its results after they are peer-reviewed. He added that the data appear to be positive, but that the system needs further testing and evaluation in clinical settings.

“Algorithms in imaging are becoming pretty effective,” Longhurst said. “But the question is how do you really implement them in a way that really helps doctors and patients.”

That is the biggest challenge facing AI in the treatment of Covid-19 and many other conditions. AI models often appear to perform well in carefully structured experiments, but the real test comes when they are deployed in actual patient care, where they must be able to perform consistently across diverse groups of patients and deliver information to doctors in a way that’s compatible with their computer systems and work routines.

“What does a community practice physician like me do with a freely available algorithm,” said Allen, the official of the American College of Radiology who practices in Alabama. “Where would I put it? How would I run it? What would the output be? The clinical integration of these algorithms isn’t there yet.”

Several private companies are working with health systems to develop and implement algorithms for the treatment of Covid-19 patients. Earlier this year, RADLogics, an Israeli AI company, struck an agreement with the Boston technology company Nuance to distribute AI applications for detecting Covid-19 and quantifying disease severity on CT scans and chest X-rays. The algorithms provide automated alerts to radiologists to help them triage patients.

Another company, VIDA Diagnostics, has developed a product called LungPrint that uses AI to analyze CT scans to help diagnose respiratory conditions, including symptoms of Covid-19. The company’s CEO, Susan Wood, said the product can be helpful in identifying patients in need of stepped up care, and helping to manage those with persistent symptoms.

“Over a third of patients are going home with symptoms such as fatigue and shortness of breath,” she said. “Those are the patients you have to watch. We can evaluate how much of the lung is still functioning and monitor that patient over time.”

VIDA is working with the University of Iowa and other partners to evaluate the tool’s effectiveness in treating Covid-19 patients.

Cohen, the researcher at the University of Montreal, said the bigger test facing these AI products is their ability to perform well in assessing patients with diverse backgrounds who live in different parts of the world. His research team has had to scrape images published in an array of international medical journals and databases to try to get enough data to adequately train its AI.

Doing so ensures the system can home in on a universal signature of the illness, as opposed to its manifestation in one group of patients. “If it works for the U.S. really well, but then doesn’t work for Canada, Germany, or France, you most likely don’t have a model that’s doing any causal reasoning,” Cohen said. “What it’s predicting from is probably specific to those American patients, and not the real thing you want.”