Successful Project Completion
After a project duration of almost 4 years, the BMBF-funded project "Development of Augmented and Virtual Multi-User Applications for Medical-Technical Exchange in Immersive Spaces" (AVATAR) was successfully completed. Together with the Clinic for General, Visceral and Transplant Surgery of the University Hospital Mainz and the two Magdeburg companies metratec GmbH and 2tainment GmbH, a new virtual learning platform for surgical training was investigated, developed and widely clinically evaluated.
Visualization of a 3D segmentation of the human liver in a virtual learning space (left), as a 3D print (center), and augmented reality representation (right). Image source: Zentralblatt Chirurgie, Thieme Verlag
Stimulated by the Covid-19 pandemic, the project placed a particularly strong focus on spatially distributed multi-user training and the topic of virtual reality. As a result, a software platform was created to allow up to 16 surgeons from different sites to come together and interact in a "virtual learning hospital." First, virtual learning spaces were developed for resection planning of liver surgery procedures. This was then followed by learning spaces for laparoscopic camera guidance, communication training for surgical complications, and surgical handoff training. In addition to the virtual learning spaces, an augmented reality solution was investigated that enables experienced surgeons to superimpose important information for training into the field of view of learning surgeons.
The results of the project provide important insights for future surgical learning products. Key project publications and videos are listed below:
Chheang, V., Saalfeld, P., Joeres, F., Boedecker, C., Huber, T., Huettl, F., Lang, H., Preim, B., Hansen, C. (2021) A Collaborative Virtual Reality Environment for Liver Surgery Planning,
Computers & Graphics, 99, pp. 234-246
Schott, D., Saalfeld, P., Schmidt, G., Joeres, F., Boedecker, C., Huettl, F., Lang H., Huber. T., Preim. B., Hansen, C. (2021) A VR/AR Environment for Multi-User Liver Anatomy Education,
Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), Lisbon, Portugal, pp. 296-305
Chheang, V., Fischer, V., Buggenhangen, H., Huber, T., Huettl, F., Kneist, W., Preim, B., Saalfeld, P., Hansen, C. (2020) Towards Interprofessional Team Training for Surgeons and Anesthesiologists using Virtual Reality, International Journal of Computer-Assisted Radiology and Surgery, 15, pp. 2109-2118
Huettl F., Saalfeld P., Hansen C., Preim B., Poplawski A., Kneist W., Lang H., Huber T. Virtual Reality and 3D Printing Improve Preoperative Visualization of 3D liver Reconstructions-results from a Preclinical Comparison of Presentation Modalities and User's Preference. Ann Transl Med. 2021 Jul;9(13):1074
Boedecker, C, Huettl, F, Saalfeld, P, Paschold, M, Kneist, W, Baumgart, J, Preim, B, Hansen, C, Lang, H, Huber, T (2021). Using virtual 3D-models in surgical planning: workflow of an immersive virtual reality application in liver surgery. Langenbecks Arch Surg, 406, 3:911-915
Heinrich, F., Huettl, F., Schmidt, G., Paschold, M., Kneist, W., Huber, T., Hansen, C. (2020) HoloPointer: A Virtual Augmented Reality Pointer for Laparoscopic Surgery Training, International Journal of Computer-Assisted Radiology and Surgery, 16, pp. 161-168