ASOURCE Navi

VR (Virtual Reality) is a technology that allows you to experience the virtual space drawn by CG by wearing dedicated devices such as goggles and headsets, and is called "virtual reality". Medical care is one of the fields where VR is being introduced, and various new attempts are being made in training, surgical support, telemedicine, rehabilitation, and so on. Here are some examples.

In the medical field, 3D image data such as CT and MRI are used on 2D screens. However, the information that can be obtained from a two-dimensional screen is limited, and a three-dimensional spatial understanding is required, especially in preoperative planning.

Holoeyes makes full use of technologies such as VR / MR (Mixed Reality), presents 3D data such as CT / MRI in a three-dimensional space, and wears VR / MR goggles so that organs and blood vessels can be observed from all angles. (Fig. 1). This makes it easier to understand the three-dimensional relationship of the anatomy, the extent of the lesion, and the depth, which were difficult to see until now, leading to support for smooth, safe, and accurate procedures. Cancer and Infectious Diseases Center Tokyo Metropolitan Komagome Hospital Surgery (Hepatobiliary Pancreas) uses VR in combination with laparoscopic surgery for pancreatic cancer, and confirms the tumor site on a stereoscopic image in the pancreas where blood vessels are intricately invaded. However, surgery is in progress.

In addition, a service called "Virtual Session" has been added, which allows multiple medical staff to share 3D models in real time in real time in virtual space and real space, and to perform treatment planning and procedure training. .. In July of this year, the operating room of Emirate specialty Hospital in UAE Dubai and the operating room of NTT Medical Center Tokyo in Tokyo will be connected online in real time, and XR (Extended Reality) remote surgery for laparoscopic gastrectomy will be performed in a complex real space. Support was provided.

Some of these functions have been certified as managed medical equipment (class II) as medical image processing software "Holoeyes MD", and will be released nationwide as a program for general-purpose diagnostic imaging equipment workstations from June 2020. I am.

Figure 1: "Virtual session" where 3D models can be shared simultaneously in virtual space and real space between surgical staff.

The research group of the Department of Neurology, Juntendo Clinic, Juntendo University School of Medicine has developed the world's first 3D online medical care system "Holomedicine". The system projects 3D motion information of a remote patient onto a remote doctor's head-mounted display (HoloLens), allowing the patient to be examined as if they were in front of them.

Holomedicine uses a technology called MR to scan 3D motion information in real time using a markerless 3D motion scanner (Kinect v2) for patients in remote areas, and a head-mounted display of a doctor in a remote location. It is a mechanism to project to. If you ask the patient to wear HoloLens and make a voice call, it will feel like they are actually facing each other (Fig. 2). The score of motor symptoms (UPDRS-Ⅲ) when treated with Holomedicine in 100 patients with Parkinson's disease has a high correlation with the score obtained by face-to-face evaluation so far, and instead of face-to-face treatment. I found that it could be. Until now, online medical care has had the drawback that it is difficult to obtain information such as walking and posture because only two-dimensional information can be obtained.

In the epidemic of new coronavirus infection, 3D online medical care system is expected as a new online medical care with an eye on post-corona.

Figure 2: The doctor side (right) and the patient side (left) appear to be in front of you through the head-mounted display.