GWU Hospital and Novarad develop AR surgical planning

12 Sep 2019

Hospital is first in US to use AR to help plan a surgical operation, improving accuracy and shortening operating and recovery times.

George Washington University Hospital (GWU), Washington, DC, in partnership with company Novarad, have reported the first implementation in the U.S. of an augmented reality (AR) medical solution to aid in surgery.

The OpenSight AR System, developed by Novarad and powered by Microsoft’s HoloLens system, renders 2D, 3D and 4D images of patients interactively, while accurately overlaying them directly onto the patient's body – including adjustment as the patient moves.

The partners say this development, “ushers in a new world of pre-operative planning, overcoming critical pre-surgical and surgical obstacles to minimize patient morbidity and mortality.”

OpenSight, co-created by Dr. Wendell Gibby, Novarad CEO, and Dr. Steve Cvetko of OpenSight, has been designed to visualize 3D imaging holograms of the patient, on the patient, for pre-operative localization and planning of surgical options. The 3D holograms are created by a refractory system in the device using a combination of the Microsoft HoloLens hardware and OpenSight’s technology.

"OpenSight allows the surgeon to overlay the medical image onto the patient's body, so not only do I see the patient laying on top of the table, but I can see the CT hologram," commented Dr. Babak Sarani, professor of surgery and emergency medicine and director of the center for trauma and critical care at George Washington University.

Visual tracking

The infrared camera within the headset has ranging and localizing technology, which maps the surrounding environment - including the patient. It determines where objects are and creates mesh surface maps to determine 3D positioning.

Placing visual tracking tags on the patient follows movement and allows for even higher accuracy. Image capabilities include virtual incisions to guide the surgeon, shunts, and needle placement; finding the optimal entrance and trajectory for pedicle screws, ACL tunnels and other instrumentation; finding and resecting masses; interventional biopsies, and more.

"A revolution in surgery is beginning and George Washington University is leading the charge," said Dr. Gibby. "Using the OpenSight augmented reality device from Novarad, doctors can literally see the patient and see inside them at the same time.

“High resolution advanced holographic imaging is merged in 3D onto the patient with optical precision. This transformative technology promises to improve outcomes in a world of more precise medicine."

The technology available through this partnership will make it possible to highlight relevant anatomical and critical structures to avoid and more accurately plan a surgical approach. Multiple headsets can be shared among users improving the training of less experienced residents. A teaching version of the software is also available that allows medical students to perform virtual dissections on cadavers.

"Novarad's OpenSight system uses the Microsoft HoloLens headset and augments real-world health care environments with virtual 3D patient images, and we are very pleased that they have achieved FDA medical clearance for pre-surgical use of their OpenSight solutions for HoloLens," said Matt Fleckenstein, Senior Director of Marketing for Microsoft HoloLens at Microsoft.