05 Apr 2018
University of Twente develops smartphone-based AR method for displaying internal features of patient body.University of Twente, Netherlands, is developing an economical smartphone technology based on the usage of Simultaneous Localization And Mapping (SLAM) and Augmented Reality (AR). This technology enables medical personnel to reconstruct 3D body sections quickly, by directing the smartphone to the area of interest. This smartphone will display the augmented layers of the 3D skin surface in order to visualize otherwise invisible information for medical personnel.
“We would like to layer the information from the MRI and other vein scans, derived from sensor devices using laser speckle, and make them visible with augmented reality”, commented Dr. Eng. Beril Sirmacek, from Twente’s Computer Science department.
“This approach allows doctors to view the inner body segments, including features such as tumors, veins and the status of diabetic ulcers with the use of augmented reality,” Dr Sirmacek added.
“This visual support enables doctors to oversee the situation better and make better decisions. In a biopsy situation, this visual support can help with guiding the robot arms to reach the tumor for biopsy at the first attempt, instead of taking the off-chance and reach for the correct location by working with a patient for a whole day and making unsuccessful biopsy holes on the patient's body.”
The augmented visualization of big data and the guidance with the smartphone application highly relies on the technology of Simultaneous Localization and Mapping (SLAM).
Dr Sirmacek added, “We are developing our 3D reconstruction and AR technology (SLAM toolbox) for smartphones, because in this way we can make sure that our algorithms can be used and help to doctors all around the world without needing to order new extra equipment. We want to provide a low cost and easy to access solution, so the developments can help to larger groups of users.
"We want our SLAM toolbox software solution to support different cameras like a plug and play system. So, this allows users to use higher resolution cameras when needed. After medical approvals are obtained for our SLAM toolbox, we would like it to be commercialized as soon as possible.”
“When we started this project, our goal was to develop a SLAM box at the UT research group RAM (Robotics and Mechatronics). We can use our localization and 3D reconstruction software on different sensors and not only with the use of smartphones. Our vision will lead to many health applications, and other robotics applications will become more intelligent. With this technology, the algorithms quickly understand the 3D models of the surrounding objects regardless of the imaging sensors which are used.”
Prof. Dr. Ir. Stefano Stramigioli, Chairman of the RAM Group, Robotics and Mechatronics, commented, “SLAM is a core technology in robotics, but its universal usage will only be possible if we have methods to easily interface many sensors in a robotic system. At RAM, we intend to create a modular SLAM box, which will automatically reconfigure itself if extra sensors are attached in a Plug&Play fashion. Then it would be possible to connect this “SLAM Box” to a complete robotic system, giving it powerful perception capabilities.”