At the MIT Reality Hack 2020, we set out to make virtual embodiment more readily available and make it easier for licensed therapists to establish a connection with their patients. Virtual Embodiment has been proven to be an effective clinical and therapeutic application of virtual reality as a therapy (Bourdin, P. 2017).
We designed an out-of-body experience (OBE) that can be controlled by a patient’s therapist. It consists of two parts: a VR application to present the OBE to the user, and an augmented reality user interface built with the Project North Star headset for the therapist to control and monitor that experience.
While my other team members focused on the virtual OBE experience using an Oculus Quest, I built an AR interface for the therapist using the Project North Star headset. It employs an Intel Real Sense sensor for 6DOF tracking and a Leap Motion sensor for hand tracking.We chose such an interface to allow the therapist to always have visual contact with the patient at all times. It would then translate into controlling the flow of the experience, such as triggering the out-of-body experience or slowing down the experience to tailor it to every patient’s needs.
Challenges
We first tried to set up the North Star headset using the newly released SteamVR drivers and using a Vive tracker to track head position. This proved to be very difficult, with problems arising from the tracker’s transform being off as well as SteamVR failing to build on our computer. We eventually switched to the RealSense camera, which took us only a half hour to setup and worked wonderfully.
Triggering animations based on Leap Motion APIs proved difficult. We tried to use the Doozy UI library to help configure these actions, but we were unable to install the Unity Package along with the Leap Motion plugin. We resolved this by writing code to sync the animations with the Leap Motion’s Slider state.
Future Work
We can use other sensors to gather biometric data (HRV, EEG, etc.) and let the therapist see more details of how the OBE is progressing. It is possible to make sure that the patient is stable enough in order to move to the next stages of the experience. There is much more room to allow the therapist to configure the experience before the patient puts on their headset. What will be the environment like? What will be the therapist’s avatar in the VR world?
Citations
Bourdin, P., Barberia, I., Oliva, R., & Slater, M. (2017). A virtual out-of-body experience reduces fear of death. PLOS ONE, 12(1). https://doi.org/10.1371/journal.pone.0169343
