Abstract
Mixed reality (MR), a technology which supplements the real world with virtual objects, is increasingly becoming available as a teaching tool in medical education. The Microsoft HoloLens device allows operators to experience MR using a head-mounted device without interfering with their physical reality, stimulating a realistic learning experience using virtual objects. This project aimed to develop a MR anatomy teaching application with HoloLens for exploring the anatomy of the temporal bone. The educational application was developed from a multidisciplinary collaboration between undergraduate and postgraduate students across several academic disciplines with Medtronic, a medical technology company. 3D anatomical models were built using ZBrush and Blender, while the HoloLens1 application was developed using Windows 10, Visual Studio 2017, Unity and Mixed Reality Toolkit (MRTK). Modules developed within the application included a basic HoloLens tutorial, a virtual temporal bone with surgical anatomy landmarks and free drilling of the temporal bone. The basic tutorial allows the operator to adapt to the MR environment prior to exploring the anatomical landmarks of the 3D temporal bone. The free drilling of the temporal bone using vertex displacement and texture stretching replicates a real-time bone drilling experience and allows the operator to explore the anatomical relationship between different otological structures.
| Original language | English |
|---|---|
| Pages (from-to) | 17-26 |
| Number of pages | 10 |
| Journal | Journal of Visual Communication in Medicine |
| Volume | 43 |
| Issue number | 1 |
| DOIs | |
| Publication status | Published - 2 Jan 2020 |
Bibliographical note
Funding Information:The authors would like to acknowledge Stefanie Schultz, Naomi Culpin and Charlie Ritchie for the assistance they provided during development of the mixed reality application. Further thanks to Danny Verkissen, Nicolas Denervaud and Agnieszka Strzembosz from Medtronic for funding the project and providing technical support and relevant resources. Lastly, many thanks to Gareth James from COMBlox Ltd for providing technical advice and support on the Microsoft HoloLens software. Materials in this manuscript were orally presented in the Anatomical Society Winter Meeting in 2017.
Funding Information:
Medtronic Public Limited Company (to P.M., P.S., R.P. and L.D.) to develop a mixed reality educational tool using HoloLens. The authors would like to acknowledge Stefanie Schultz, Naomi Culpin and Charlie Ritchie for the assistance they provided during development of the mixed reality application. Further thanks to Danny Verkissen, Nicolas Denervaud and Agnieszka Strzembosz from Medtronic for funding the project and providing technical support and relevant resources. Lastly, many thanks to Gareth James from COMBlox Ltd for providing technical advice and support on the Microsoft HoloLens software. Materials in this manuscript were orally presented in the Anatomical Society Winter Meeting in 2017.
Publisher Copyright:
© 2019, © 2019 The Institute of Medical Illustrators.
Keywords
- educational tool
- ENT
- HoloLens
- mixed reality
- surgical
- Temporal bone
ASJC Scopus subject areas
- Health Professions (miscellaneous)
- Visual Arts and Performing Arts