The US Food and Drug Administration (FDA) is hosting a public workshop entitled “Medical Extended Reality: Toward Best Evaluation Practices for Virtual and Augmented Reality in Medicine”. According to the agency’s website, the purpose of the public workshop is “to discuss evaluation techniques for hardware, standards development, and assessment challenges for applications of Extended Reality (XR) in medicine. The goal is to identify critical gaps that may impede medical XR device development, innovation, and to advance the evaluation of medical XR devices and applications, thus accelerating the development of safe and effective medical XR devices benefiting patients and healthcare.” Follow for live updates from Interventional News here.
13:41 GMT
Edward Margerrison (director, Office of Science and Engineering Labs, Center for Devices and Radiological Health, FDA) welcomes attendees. “I want to set the scene a bit, and explain why the FDA is involved,” he says. “We have representatives here from industry, from academia, we have healthcare professionals, and we have members of government as well. So we really have the whole community here.
“Much of the technology has been developed outside of healthcare, outside of medical devices, and is beginning to come into FDA to be regulated. It is a very interesting area for us, because companies are developing products that haven’t previously been in a regulated environment. Now is the time to get everyone on the same page, to have real discussions about how we are going to develop these technologies, and stimulate innovation.
“We are looking in a pre-competitive space. […] The technologies and products of the future should compete on how good they are for the customer. […] Being the FDA, a lot of people look to us as leaders. I anticipate a lot of people asking today, is this the end of the beginning, or the beginning of the end? We think it really is the end of the beginning. Particularly the AR/ VR space (augmented reality/ virtual reality), we know it is coming, but what are the things that we do not know yet? We want you to explain so we do not become a barrier.”
13:49 GMT
Technical session begins.
“At the end of the day, at around 4pm [ET], we will need to come up with ideas for how to move forward, and we want those ideas to come from you,” Aldo Badano (deputy director, DIDSR, OSEL, CDRH, FDA) says, encouraging attendees to “be bold” and come up to the microphone to ask questions and air opinions.
“With the exception of our keynote speaker, we do not want to focus on the hype or what would be possible if we had infinite amount of time and resources, but rather what are the current challenges that are impeding the application of these technologies in the medical field.”
13:55 GMT
General trauma surgeon Rafael Grossmann (Maine, USA), the first surgeon to use Google Glass, starts his keynote lecture.
Talking of VR, Grossmann enthuses: “It blows my mind, what the technologies allows you [to do] today. What you can see through s VR headset is incredible, and this is just the beginning. The future of this technology is to become more ergonomic, more economically viable, and, ultimately, much more common.
“We are going to see a paradigm cost in the cost, weight, size of these devices that will transform the way we do health care.”
He shares BrainLab, a company he claims is “revolutionising how we educate patients and providers in regards to surgery and planning”. A representative from BrainLab will speak later. Grossman stresses the importance of haptic feedback when using VR or AR in surgery, and says that BrainLab is a “pioneer” in this field.
Grossman is currently not using VR headsets clinically, but for teaching purposes. “This is not a matter of sterility, there is no infectious concern, but we have not found the right combination of this technology and procedure in the operating room. We are still waiting on regulations.”
14:23 GMT
Bernard Kress, an optical engineer at Microsoft, takes to the podium to present on advances in mixed reality devices and new evaluation challenges.
He says HoloLens is not advertised for general consumers, but is specifically for healthcare providers. The legal team at Microsoft requested Kress read the following disclaimer ahead of his talk: “HoloLens is not intended to be a medical or surgical device, but our clients say it can be effectively used in medical imaging and image assistant surgical planning.”
Several companies, including Philips, are already using HoloLens in the medical space, Kress comments. Atul Gupta will speak on behalf of Philips later in the programme.
“Everything that we do is centred on the human,” Kress explains. “The first pillar of mixed reality is immersion. Immersion has to be matched with what the human can actually sense. This is very difficult. You do not want to over-design your system; it has to fit what the actual human can see. You have to match the optical resolution and the field of view with that of the human eye.”
The second “pillar” is comfort. “This is especially important if the surgeon has to wear the headset for multiple hours”. There are four types of comfort Microsoft take into account: wearable comfort (taking into account size, weight, temperature, centre of gravity), vestibular comfort (the device has to be physically untethered, and a large, transparent, unobtrusive peripheral see-through is required so wearers are aware of their surroundings), visual comfort (having non-conflicting 3D cues and the shortest “motion to photon” latency possible), and social comfort (need to be able to make eye contact with nurses and technicians in the room, and not put off the patient).
“So where are we today with MR hardware?” Kress asks. “There is still a long way to go to provide full comfort to surgeons and medical professionals to allow for a revolutionary imaging and augmentation tool which will change forever the medical imaging field. This is just the tip of the ice berg.”
When asked by an audience member when the cost of MR headsets will go down (“Like VR headsets, which can be very cheap”), Kress says “It will take a long time. There is no AR market today for consumers, so we are not even looking at a consumer price point. For industry, right now, I think the device is around US$3,500. This will go down, but we are not constrained by consumer requirements.
“However, this ecosystem of companies that are trying to reduce the cost of these very complicated optics are emerging, so in five to ten years I think you will see a price decline of a degree of magnitude.”
14:52 GMT
Hiroshi Mukawa, of Sony, speaks on “Emerging technologies for addressing performance limitations in extended reality devices, such as motion-to-photon latency compensation, and retinal scan displays”.
15:09 GMT
Vinay Narayan, from HTC, delivers his talk, “Advances in virtual reality devices and characterization methods”.
“We started off in VR,” he says, “but the ability to build products that can scale in a meaningful time frame really made us switch our priorities to the VR space.” Echoing Kress, Narayan stresses that one of the key differences between AR and VR when thinking about commercialisation is that VR products are available for general consumers.
17:05 GMT
John Penczek, from NIST, takes to the podium to discuss progress in standards for extended reality devices. Providing a review of the International Organization for Standardization (ISO), he outlines how the standard-setting body has robust, specific guidance for consumer requirements for finding eye point, but that there is still more work to be done, in particular in the realm of eye tracking and direct retinal tracking.
18:45 GMT
Atul Gupta, from Philips, outlines the opportunities and challenges for extended reality devices in medicine.
