L

A JOLLA, Calif. — Spend enough time with Larry Smarr and, chances are, he’ll invite you to step inside his colon.

Like more than a million Americans, Smarr has inflammatory bowel disease. Unlike most, he also runs a cutting-edge institute replete with reams of ultrafast computers, crack graphics programmers, a towering wall of digital screens and a pitch-black virtual reality cave — all the better to summon up a digital 3-D version of himself that he calls “Transparent Larry.” Among its features is a larger-than-life replica of his colon that includes every nook, cranny, and section of inflamed tissue.

Smarr, 69, is a physicist widely recognized for his work on creating the national network of campus supercomputers that evolved into today’s internet. Now, he runs a futuristic institute called Calit2, housed on the University of California campuses in San Diego and Irvine, that works to advance a host of fields, including medicine. For the last decade, he’s been turning technology on to himself to quantify his body’s most intimate workings, with no clear idea where the experiment might lead.

But when his health started to fail and surgery was indicated, Smarr saw the perfect opportunity to use his hoard of big data and pioneering visuals to inject a much-needed software upgrade into the operating room. He’s hoping his institute can help create the automated software that might make 3-D images in surgery routine within a few years.

His trial run — with himself as guinea pig — started a few days before his surgery last November, when Smarr called his colorectal surgeon, Dr. Sonia Ramamoorthy, to his institute’s VR cave.

“I thought, I know more about the particularities of my insides than my surgeon does,” he said. “That can’t be good.”

Newsletters

Sign up for our Daily Recap newsletter

Please enter a valid email address.

Ramamoorthy wasn’t surprised by the slightly odd invitation; nothing about her relationship with Smarr has been normal. “Unlike any patient experience I’ve ever had,” she said, “he came equipped with a PowerPoint.”

It turns out Smarr had detected a glaring omission in today’s modern operating suite: The 3-D computer visuals that are standard in even the most basic of video games are not routinely available to surgeons — despite their utility in detecting unexpected anatomical variations that can derail surgeries.

A three-dimensional animated model of Larry Smarr’s affected colon (green), with spine (white), spinal disks (yellow), major blood vessels (blue and red), bladder (light blue), and spleen (magenta). Courtesy of Jürgen Schulze, UC San Diego

When they have 3-D imaging available, “surgeons love it,” said Eric Wickstrom, a professor of biochemistry and molecular biology at the medical school at Thomas Jefferson University in Philadelphia. He has worked for more than a decade to develop 3-D imaging for use in cancer surgery and published a 2013 study concluding that surgeons found it useful.

“They say, ‘Our biggest problem is cutting into an artery or vein that we did not expect,’” Wickstrom said.

“I thought, I know more about the particularities of my insides than my surgeon does. That can’t be good.”

Larry Smarr

But Wickstrom said very little 3-D technology has been deployed to operating rooms because it’s still too time-consuming to create the lifelike images from a stack of normal, 2-D images of the patient’s body. More research and programmers are needed, he said, to automate and speed the process.

Smarr’s 3-D model evolved over years and required much painstaking work from virtual reality expert Jürgen Schulze: He first had to go through a stack of 2-D MRI slices outlining each organ. Then he had to combine them to create a 3-D model fit for viewing on a regular computer monitor. Automating this step is a critical process in making it easier and cheaper to use such images routinely.

A surgeon visits the virtual reality cave

When she stepped inside the dark cave and popped on a pair of 3-D glasses, Ramamoorthy was amazed. Imaging of the abdominal area has lagged behind other fields because that section of the body houses so many organs of different densities and because it moves with every breath and every contraction of the intestinal muscles.

Ramamoorthy immediately saw a “no fly zone” where Smarr’s inflamed colon was sitting smack on top of his bladder. One nick from a surgical tool and urine could flood the abdominal cavity. (Luckily, they weren’t connected.) She also noticed Smarr’s large intestine rose much higher than expected and was adhered to his spleen.

The veteran surgeon knew instantly she’d have to change her surgical game plan — and knew she wanted those 3-D images with her in the OR.

“The 2-D images, they’re just not that helpful. They don’t help me think in 3-D,” Ramamoorthy said. Having a trove of 3-D images she could see in advance and even move through in virtual reality, she said, “was like a candy store for me.”

But when Ramamoorthy asked Schulze if he’d bring the 3-D images into the OR, he blanched. “I’m not good with blood,” he said.

In the end, Schulze agreed to come.

“He manned up,” said Smarr.

The day before the operation, Schulze was in the operating room, fiddling with cables, trying to see if he could connect his laptop to the surgical robot and project the images onto screens at the Jacobs Medical Center, a $943 million hospital that opened just days before Smarr’s surgery. The new OR suites, built to handle lasers, robots, and MRIs that can be used in the midst of surgery, seemed the perfect home for the radical new imagery.

And it all worked. While Ramamoorthy manned the four-armed da Vinci surgical robot anchored over Smarr’s body, others in the room had their eyes glued to an 85-inch, wall-mounted screen. It was divided into four views and showed Smarr’s virtual colon in 3-D, his MRI, a view into his abdominal cavity from above the robotic arms, and another close-in view from inside the colon, also generated by the surgical robot.

“I look at my teenaged sons and their Xboxes. All of that technology is there — why is it not there for my patients?”

Dr. Sonia Ramamoorthy

“It looked like the Starship Enterprise,” said Smarr.

Ramamoorthy likened the visuals to driving with Google Maps. Instead of having only a vague idea of where you might be, she said, “you know where you are, how far along you are and if there are any accidents ahead.”

The top-notch 3-D visuals have been the talk of surgeons at the hospital. And now that she’s had a chance to use them on one patient, Ramamoorthy keeps wishing — especially on the tougher surgeries — she could have them for others.

“I look at my teenaged sons and their Xboxes,” she said. “All of that technology is there — why is it not there for my patients?”

Dr Larry Smarr
Smarr tracks 72 data points about his health and displays it on a wall in his institute covered with 32 screens. Sandy Huffaker for STAT

A couch potato discovers kale

Smarr’s obsession with learning everything he could about his body started nearly two decades ago — with a diet.

Smarr had spent two decades in Illinois, directing the National Center for Supercomputing Applications, before he was recruited to UCSD in 2000. He arrived to the seaside campus pasty, overweight, and surprised by the healthy lifestyles that surrounded him.

“I’d never eaten kale before,” he said. “I was afraid they would send me back.” Smarr immediately got a personal trainer, went on the Zone diet, and started taking nutritional supplements.

Smarr lost 30 pounds and got his 6’1” frame into shape. (He still keeps his old driver’s license in his wallet to remind himself what he used to looked like.) Not one to do anything lightly, Smarr wanted to know if he was taking the right number of fish oil capsules for optimum health. So he started taking blood tests to check and then got a little bit addicted to all the data he could collect. He wanted more and more.

“It was like a loose thread on a sweater. I just kept pulling on it,” he said.

There were many roadblocks: He had to fight with doctors who told him that he shouldn’t order his own blood tests, but that they wouldn’t order them either: “They said there’s no insurance code for that. It’s preventative.” He paid for the blood tests out of pocket — and estimates he’s spent tens of thousands of dollars by this point. “We talk about patient-centered medicine, but we’ve clearly got a long way to go,” he said.

In 2005, even though he felt fine, his blood tests showed him something was wrong. His C-reactive protein, or CRP, was five times and then 15 times the upper limit that is considered healthy. “I had no idea what CRP was. So I go to PubMed and start googling,” said Smarr, who has no biomedical training but has served as an adviser to several NIH directors. He learned CRP was a generic measure of inflammation.

It was telling him something was seriously wrong with his body but he had no idea what.

Smarr didn’t suspect his colon because his 2005 colonoscopy results looked normal. It turns out that was because his disease was atypical in many ways. It was late onset, so his intestinal tract didn’t have the battlefield of scars his physicians would expect in a man of his age, and the damage was confined to a small part of the colon.

When his CRP levels shot up again in 2008, Smarr went to his physician. Smarr, who started his career as an astrophysicist, said he was promptly sent home. “They said, ‘Do you have symptoms?’ I said, ‘I’ve got data!’ If I saw something like that in the sky, I’d be training all the telescopes on it! They said, ‘Come back when you have symptoms.’”

The crushing abdominal pain started shortly afterwards. “It was totally predictive,” Smarr sighed. “I look at medicine and say, ‘What is wrong with these people?’”

Newsletters

Sign up for our Morning Rounds newsletter

Please enter a valid email address.

Ten days of antibiotics eased the symptoms, but never fully got his CRP numbers under control.

Unsettled, Smarr decided to take a look at genetic data he had from 23andMe. (Yet again, he was an early adopter.) He found an SNP, or genetic variation, that indicated he carried an 80 percent increased risk for Crohn’s disease, which can flare episodically.

A controversial obsession with data

In 2011, his numbers got really bad. So Smarr started collecting his stool samples for analysis. He’s had 90 tests at $375 a pop and is still collecting. (Don’t ask to look in his freezer.)

The computer scientist in Smarr is enamored with feces because it contains so much data, including hints about the health of the immune system and intestinal tract. “There are a billion bacteria in a gram of stool. Each has 5 million bases of DNA,” he said. “I look at it like a disk drive.”

By mid-2011, a stool analysis showed his levels of a protein called lactoferrin, which is linked to inflammatory bowel disease, were 125 times above the upper limit. By the end of 2011, meanwhile, his CRP had spiked to 27 times the healthy limit. A colonoscopy showed his colon wall — which was being attacked regularly by his own immune system — was much thicker than it should have been.

At Calit2, Smarr can graphically display a decade of his health data onto a digital wall made of 32 high-definition screens — cholesterol, glucose, insulin, vitamin D, CRP are among the 72 parameters he tracks. “This is me,” Smarr said.

And while he realizes that most people don’t have the same technology at hand, he feels strongly that consumers should take their health in their own hands and seek out data on their bodies.

“I look at medicine and say, ‘What is wrong with these people?’”

Larry Smarr

That’s a controversial view: Many doctors believe that regular blood tests for healthy people can do more harm than good.

Tests can yield false positives, after all, which can cause anxiety and drive up medical bills as the results are checked and re-checked. And while it turned out that Smarr was right to be concerned about his CRP levels, in many cases, anomalies in test results are just that — anomalies that don’t necessarily signify real health threats. If patients rush out to treat every one of them, they’re not only going to ring up big bills, but they might also do themselves harm. Treatments inevitably have side effects.

Given those risks, the Society of General Internal Medicine doesn’t even recommend annual physicals or routine lab tests for healthy adults.

Smarr disagrees. He knows people think he’s a little bit nuts. But he doesn’t care. “If I’m not 10 to 15 years ahead,” he said. “I’m doing something wrong.”

‘I felt like a balloon about to burst’

Because he was so fascinated with the data he was collecting on his disease, Smarr did not want to be treated. “I didn’t want to mess up the beautiful time series,” he said.

He was no stranger to collecting data through pain; he once worked through the night with a raging case of pneumonia rather than give up valuable observing time at the Very Large Array Radio Telescope in New Mexico. “I am a scientist,” he said. “You do what you have to do to get the data.”

But in 2012, the bloating and rectal bleeding grew overwhelming and Smarr gave in. “I’m dedicated to getting data, but I’m not stupid,” he said. He took a month-long course of antibiotics and sterioids. His CRP fell to 1.3, which is considered relatively normal — but he wanted to see it lower still. And he noticed, oddly, that his symptoms did not seem to diminish as the CRP levels fell.

“ If I’m not 10 to 15 years ahead, I’m doing something wrong.”

Larry Smarr

In 2016, a full body CAT scan (including a virtual colonoscopy where, said Smarr, tittering a bit, “they inflate you through your back end with air”) showed his intestinal walls were so thick in places that his lumen, or colon interior, was only a few millimeters wide. It was so narrow, indeed, that even a pediatric colonoscopy tool would not fit through. One section had an unusual kink. Stool was backing up and generating methane gas in his body.

“My wife said, ‘Oh my God, you look pregnant,” said Smarr. “I felt like a balloon about to burst.”

It was time for surgery.

Smarr operation
Dr. Sonia Ramamoorthy, guided by the 3-D imagery, operated on Smarr’s colon during a five-hour procedure in November 2016. Jurgen Schulze/UCSD
Smarr
Dr. Santiago Horgan,  director of the Center for the Future of Surgery, points to area of interest in Smarr’s colon on the operating room’s big screen. Jurgen Schulze/UCSD

Excising ‘the demon baby’

With the help of the virtual preview, Smarr’s surgery went smoothly. Ramamoorthy used the robot to cut and cauterize the inflamed colon, staple his intestines back together and reopen the lumen. Ramamoorthy then pulled out two large fistfuls of ugly red tissue — what Smarr calls “the demon baby.”

Luckily for VR expert Schulze, who sat through the entire procedure, there was not much blood. Smarr lost only about 15 milliliters, not much more than a typical blood draw.

One clear benefit of the 3-D imagery was that surgical time was shorter, which often leads to a better outcome for patients. Ramamoorthy estimates she shaved at least 30 minutes off the five-hour surgery by knowing what to expect in advance and saved another 45 minutes by docking the robot onto Smarr in a much higher location than usual to deal with his adhered spleen.

The surgical recovery was swift. With Smarr as a patient, however, it was not routine.

Just hours after waking, Smarr was back to collecting data. He desperately wanted to track his CRP numbers post-surgery to see how the procedure had affected his body. But the nurses told him CRP tests hadn’t been ordered by his physicians. Smarr refused to let them take his blood until they found a doctor who would authorize the CRP test along with the others they were running.

Dr Larry Smarr
Smarr shows multi-dimensional graphic detail of his intestinal tract. Sandy Huffaker for STAT

“They thought I was a loon,” he said. “A problem patient.” Aside from being neat data points, Smarr thinks CRP tests should be routine after all surgery for early detection of problems like sepsis. “It’s an $11 blood test after a $150,000 operation,” he noted.

He got his way and watched his CRP drop in the following days from 61 to 42 to 30 to under 1 for the first time in more than a decade. He also smuggled in his Fitbit and — to the raised eyebrows of staff — a team of colleagues who outfitted his lower abdomen with electrodes, hoping to detect the instant his colon started working again.

Now fully recovered, Smarr shows no sign of slowing down.

“Do you want to hold my colon?”

Larry Smarr

His goal is to create software that could automate the creation of 3-D images of organs, and he thinks imaging companies could be using such software within three to five years. He also thinks his institute can play a role in training a new generation of medical imaging VR experts.

For now, he offers digital tours of his body to physicians who ask and gives talks — sometimes alongside Ramamoorthy— about the power of 3-D visuals.

“Do you want to hold my colon?” he’ll ask as he hands out a 3-D printed replica of the organ.

He also extols Ramamoorthy’s work at every chance. “I love my new colon,” he told a crowd at a recent surgical grand rounds. “It works so great.”

Leave a Comment

Please enter your name.
Please enter a comment.

  • It occurs to me that if a camera embedded in a pill or manually guided would travel the GI, then that completely portraits expected images and attached to micro tools provides for access to medical interventions.

    Unless issues are found that require further study 3D modeled, at cellular/neural levels, and then where multi-factor interactivity would need to be captured and study, then there is a need for Medical Research.

    I believe that would be some how much more difficult to achieve if it was the brain that would be in question but 3D has reached OR.

    I would like to know how 3D printing and AI would play in achieving accuracy for a cancer level diagnostics and test during lymphatic node count evaluations?

    Blood, saliva, breath and even sweat genetic studies can give light into disease today, but accuracy is still dependent upon methods and sensor qualities.

    A virtual/augmented reality might be a reflection of one on the other: Guts Health to Brains Health?

  • Great read. Glad 3D technology is about to make it into the OR. With tech advances, more lab tests will be plotted in real time, looking at the trending.

  • The problem of converting 2D slices into 3D images remind me of that confronted by neuroscientists in mapping the structures of the brain. I believe that the algorithms developed may be quite advanced.

  • My favorite STAT article in quite some time. Taking control of one’s health. Moving beyond the status quo. Quite an endorsement for pushing the boundaries of evidence-based vs. eminence-based medicine: “I look at medicine and say, ‘What is wrong with these people?’”

  • The Zone Diet? And he seemed like a pretty smart guy. If he is smart he won’t be on the Zone Diet much longer.

Sign up for our Biotech newsletter — The Readout

Your daily guide to what’s new in biotech.