Imagine what it would be like to perform heart surgery on patients with VR (virtual reality)? In fact, such a scene appeared in the American drama the good doctor. Even medical devices are presented by VR. “Don’t think it’s just science fiction or TV drama. It’s really happening,” said Dr. David Axelrod, a clinical associate professor of pediatric cardiology at Stanford children’s medical center.
Indeed, in September 2017, the Royal Hospital of London used VR technology to treat the world’s first 360 VR cerebral aneurysm. This is a neurosurgical operation that needs to repair the patient’s two aneurysms. The doctor wears GoPro. Yes, don’t think VR can only play games. Its application in the medical field deserves our attention. Recently, Silicon Valley has insight into the application of VR technology in the medical industry. He interviewed Dr. David Axelrod, an associate clinical professor of Stanford University, and asked him to show us how VR technology is applied in the current American medical industry.
According to Dr. Axelrod, VR technology has been used to train students and educate patients since 2016, which is also the first major use of VR technology in the medical field: training and education. The training and education objects include medical students, doctors and even patients. Medical students and doctors understand better. Like Western Reserve University, Microsoft hololens is used to help students learn anatomy with VR instead of learning anatomy through cadavers. Young medical students, wearing hololens, directly “see through” the human body, from skin to muscle to blood vessels.
At Stanford University School of medicine, 400 neurosurgical patients have observed how their surgery will be carried out in virtual reality before surgery. After the operation, Stanford University School of medicine also made VR images of the patient’s brain, spine and other body parts. In the past, these parts were explained by physical models. By immersing the patient in the anatomical structure, we can clearly understand the location of the operation.
Of course, some body parts that move with the flow of blood and breathing are not involved at present. Malie Collins, senior project leader of VR program at Stanford University School of medicine, believes that this will soon expand to other parts of the body. In addition to the brain and spine, there is also the heart. Dr. Axelrod, for example, because the ICU ward of Stanford children’s medical center usually needs to deal with patients with serious heart defects, the complexity of the operation will inevitably bring fear and confusion to patients and their families. Therefore, VR plays a role in it.
“They (patients and their families) may be afraid and confused. Why do we do this (surgery). Specifically, which heart part will be operated on and how to operate. In the future, let patients understand the specific situation of surgery from vr virtual reality, so as to alleviate some fear,” Dr. Axelrod said. However, at present, this application is still in the experimental stage. It is mainly presented to patients after surgery, so that patients can better understand what they have experienced through VR.
For medical students, using VR for simulation learning is already part of a four-year course at the University of California, San Francisco School of medicine. In the first year, medical students can be exposed to virtual reality, understand human structure and experience anatomical experiments. For surgeons, they can practice each step in the virtual reality world before entering the operating room, so as to improve the accuracy.
In fact, it is necessary to use medical simulation training before operation. A study conducted in 2016 by Johns Hopkins Medical Center, the top medical center in the United States, estimated that more than 250000 people in the United States die of medical errors every year, which has become the third leading cause of death in the United States. In a 2008 survey of members of the American College of surgeons, nearly 10% said they were “worried about making major medical mistakes in the past three months”.
In addition to helping patients, medical students and doctors, Dr. Axelrod said that VR technology can also help high school students learn science and improve their interest in medicine. Stanford University has also launched such cooperation projects in local high schools. Dr Axelrod told Silicon Valley insight that in addition to VR surgery, even AR (augmented reality) was used in surgery. That is to project medical images onto the surface of the patient’s body so that the patient can see them. This is even more real than VR. “If you put a medical image in augmented reality on the patient, you can see all about anatomy and everything.”.
Explain the difference between the two technologies in the medical industry: virtual reality is to integrate users into a computer-generated virtual world, while augmented reality is to add computer-generated virtual objects to the real physical space. One in and one out. It does sound like a strong sense of technology, but why hasn’t it been popularized and even applied on a large scale in surgery? This brings us to the most difficult part of VR surgery – medical imaging.
In Dr Axelrod’s view, a major challenge now is how to convert medical images into VR, which requires high-quality imaging. It needs the help of many technical engineers. Imagine that you should first obtain high-quality medical images and then import them. It can also interact as a virtual reality. Besides the actual interaction, there should also be a safe response, such as preoperative planning before VR surgery. This series of processes are very complex and high-precision.
Can technological progress only depend on science and technology enterprises? Dr Axelrod thinks so. No matter from the perspective of hardware or software, it needs to be improved. For example, hardware devices need to be more portable and cheaper. Now with the help of actuarial computer, there is no need for a huge computer next to it, which is a great progress.
In terms of software, in addition to developing corresponding software with the support of large companies with VR business such as Facebook \ / oculus and Microsoft, many start-ups have also begun to cooperate with medical communities and have developed software that can quickly identify 3D imaging and output it. This means that in addition to the technical challenge, another challenge comes from the medical community. Dr. Axelrod believes that another major challenge in the application of VR technology in the medical field happens to come from the medical community. Because compared with scientific and technological groups, medical groups will accept a technology more slowly and establish standards more slowly. “This is not to say that it is not friendly to technology, but that it is related to human life after all. So it will be more cautious.”
“It takes time, but that doesn’t mean we can’t get there,” Dr. Axelrod said optimistically. When it comes to the medical industry, an indispensable role is the FDA (U.S. Food and Drug Administration). So, what role does FDA play in the application of VR technology in the medical industry? Dr. Axelrod pointed out that FDA is indeed an important role, but in the medical industry, FDA is not only a regulatory role in licensing, but also a partner and advocate in many cases.
In Dr. Axelrod’s opinion, many medical associations have begun to cooperate with FDA, and FDA is also very easy to hear the work of medical associations. “Only by knowing more about what they (medical associations) do and ensuring that things are done safely and considerately, FDA will raise good questions and consider the whole industry. Therefore, it is more like a collaborator and advocate.”
Last October, the U.S. Food and Drug Administration granted opensight, a product of novarad, a medical imaging company, a 510 (k) license, which is the first medical ar solution approved by FDA in the United States. It mainly uses Microsoft hololens’s augmented reality (AR) to build a system called opensight, which is used for preoperative planning of surgery. However, so far, Silicon Valley insight has not found that medical solutions related to VR have been approved by FDA.
In this case approved by FDA, opensight system can project and cover the 2D, 3D and 4D images of the patient’s body on the patient’s body. Doctors combine the scenes inside the patient’s body and the information outside the body through hololens to increase the understanding of the operation plan and anatomical relationship, so as to reduce the operation time. According to the data of ABI research, a consulting firm, VR services in the field of medical and healthcare will grow explosively from US $8.9 million in 2017 to about US $285 million in 2022. Behind the growth in numbers are more and more application scenarios of VR technology in the medical field.
Professor Walter Greenleaf of the VR laboratory at Stanford University is also a neuroscientist. He summarized the application of VR technology in the medical field into the following four categories: health and wellness, professional training, clinical interventions and clinical diagnosis.
For example, VR is used in the field of health care, such as meditation and decompression. Another major application field is training and teaching, which is also mentioned above. For VR technology, the third application scenario is medical intervention. VR technology is applied in a large proportion in the field of medical intervention, and it has started earlier. According to Dr. Axelrod, this is mainly aimed at some neurological diseases, trying to intervene and treat diseases through VR environment. For example, VR technology can help burn patients relieve the pain caused by treatment, and PDST patients (post-traumatic stress disorder). Because patients with PDST are usually in a state of depression, anxiety, or overreaction for a long time.
In a study conducted by the military for soldiers injured and burned by improvised explosive devices, VR was even better than morphine. According to patient reports, the use of VR reduced pain by 60% to 75%, compared with an average pain reduction of about 30% with morphine. Last year, Samsung Group also cooperated with local hospitals in Korea to develop a mental health diagnosis plan using VR. Among them, the cognitive behavioral therapy of suicide prevention and psychological evaluation will become the focus. In other words, at present, according to the difficulty of application, clinical diagnosis will be a major link that needs to be tackled collectively by the medical community.