RESEARCH PORTFOLIO
A major challenge in the medical extended reality space is simultaneous training of multiple users across distances of medical diagnostic and procedural skills. To address this challenge, we are developing an open-source software development kit (SDK) platform to run simulation course content on the platform. The platform allows for both AR and VR training, and will have the capability to run sessions at multiple sites simultaneously.
COLLABORATIVE SIMULATION PLATFORM
OUTCOMES AND IMPACT ON INDUSTRY
The SDK platform allows industry members to collaborate and run simulations on the platform. In addition to the SDK platform itself, we are creating a toolbox to develop simulation use cases, and to demonstrate that the platform is useable between MIXR partner sites and the AFRL, then will generate pilot data.
QUANTIFYING LATENCY & CYBERSICKNESS IN XR
As cybersickness is a key obstacle to the full deployment of VR environments, understanding the phenomenon and how to mitigate it is at the root of all aspects of immersion. Building on our previous MIXR work, we are validating our EEG-based measurements of cybersickness, and relating these measures to the optical flow of a given scene. (i.e. how much the scene changes from frame to frame).
OUTCOMES AND IMPACT ON INDUSTRY
Our EEG studies provide industry members metrics for measuring cybersickness, and offer an opportunity to vastly improve end user experience. Using open-source software, we are incorporating our cybersickness measures into a benchmark for assessing the likelihood of a given immersive scenario causing cybersickness.
ASSESSMENT OF HOW LATENCY AFFECTS PROCEDURE PERFORMANCE
A lack of standardized measures of how XR headsets affect medical procedure performance remains a major gap. We are using ultrasound-guided needle biopsy of a breast mass (a short, yet technically difficult procedure) to study how latency and other visual artifacts of an XR headset affect procedure performance. Metrics studied include physiologic responses (heart rate, heart rate variability, and galvanic skin response), procedure success, error rate, and performance time.
OUTCOMES AND IMPACT ON INDUSTRY
Improved assessment of medical procedure performance in XR headsets broadens industry member reach into medical environments. We are performing a pilot study with medical students and first- and second-year surgical residents with minimal experience in doing breast biopsies to assess the test bed, to inform a larger randomized cross-over study.
STANDARDIZATION OF XR INTEGRATION IN HEALTHCARE FACILITIES
Security and implementation barriers are a significant roadblock to widespread adoption of XR in hospitals and medical schools. To standardize XR integration, we are compiling the various workflows for onboarding XR technologies planned for utilization in both clinical and non-clinical settings. Devices and applications undergo a standard process, with transparency between the IT infrastructural teams and the medical teams providing the implementation.
OUTCOMES AND IMPACT ON INDUSTRY
MIXR standardization efforts promise to significantly broaden industry member access and reach into medical environments. We are developing an agreed-upon set of standards between partner institution IT teams, a proposed broader strategy/process for integration of XR technologies into the healthcare institution infrastructures, and a proposed set of recommendations for industry on rapid integration with healthcare institutions.
EVALUTION METHODS FOR MARKERLESS REGISTRATION
Image registration methods for surgical applications raise safety and effectiveness concerns compared to more traditional frame-based techniques. To address this for the specific case of extra-corporal membrane oxygenation (ECMO) cannulation, we conducted a background survey with experts to define the steps of an XR-based training course in this procedure. To build the camera-based AR tracking application, we video recorded ECMO training, acquiring footage to be used in markerless tracking software.
OUTCOMES AND IMPACT ON INDUSTRY
Validating markerless image registration offers industry members a path to creating and deploying multiple general surgical scenarios. We can recognize and track targeted medical instruments for ECMO cannulation procedures (i.e. cannula, ultrasound probe, and syringes) in 3D space. Three different lighting conditions were tested to determine the adaptability and quality of the tracking software, as well as account for unpredictable lighting in the real world.