Virtual Reality is the Latest Tool for Surgeons

For the Martinez babies, the VR experience allowed the doctors to “walk into [their] heart,” said Dr. Arthur Erdman, the director of the Earl E. Bakken Medical Devices Center at the University of Minnesota. The center is one of the leading locations for innovative medical tools, such as 3D printing, virtual reality and other cutting edge techniques.

“When you get into complex situations, even the most skilled radiologists and surgeons can benefit from being able to scan through the anatomy in whatever direction they want in real time,” he told i3. “They can enlarge it, turn it around and try to understand the complexities.”

To do this, the team at the Medical Devices Center enlisted Ph.D. students and colleagues to manually combine CT scans to craft a 3D model of the connected hearts. By late April, surgeons used VR glasses to view the model as if it was right in front of them. 

While wearing them, each doctor in the room could see the twin’s connected heart, at the same time. As they looked into the detailed digitization, they explored and studied the model, planning each step of the surgery as they examined it, Erdman said. 

But they found something shocking. The scans pinpointed a 1 millimeter vessel connecting the twins’ hearts that would need to be cut. 

“If they hadn’t realized that was there, it might have been devastating during the separation,” he said.

Thanks to these cutting edge techniques and world class doctors, the surgery was successful – Today, the twins are separated and recovering. 

VR and Beyond

Erdman first envisioned using an interactive VR system after a 2006 conference for the American Society of Mechanical Engineers and the Food and Drug Administration. By combining 3D models with interactive displays and a user interface, Erdman realized it could help engineers and physicians optimize medical devices and better understand patient-specific anatomy. To Erdman, VR belongs in the same toolbox as a medical textbook or an X-ray machine. Preparing surgeries with 3D objects, not just 2D renderings or images, has an obvious advantage. 

Also in that toolbox: 3D printing, which doctors also used to help the Martinez babies. The University of Minnesota’s Visible Heart Laboratory and the Medical Devices Center 3D printed a number of physical versions of the twin’s combined hearts to see exactly how the girl’s hearts connected. The MDC will soon upgrade to a printer that can print six different materials, Erdman said – the current one can print two. 

Now, the industry, is even starting to take notice – VR is also being used to educate patients what to do during procedures. Through immersive learning, medical students can see firsthand what steps need to be taken. One platform even livestreamed an operation in a 360° viewpoint of the doctor. 

As time goes forward, Erdman hopes VR technology becomes more commonplace to help doctors prepare for complex surgeries. The Medical Devices Center, for example, already is developing a second generation system that can project images “like a hologram.” 

“In future years, this [technology] will end up in all surgery suites,” Erdman said. “You’ll have the ability to pull up images with voice command and be able to pull up patient info just as easily.”