The advent of Augmented Reality (AR), Virtual Reality (VR), and Extended Reality (XR) technologies has significantly impacted various sectors, prompting substantial investments in related software and hardware. Among the promising applications of XR is its potential to enhance training in fields such as medicine and industrial engineering. Traditional training methods often fall short in effectively conveying complex physical tasks, leading to inefficiencies and reliance on experienced instructors. This paper introduces an innovative XR-based training application that builds on the foundation of 4D Holographic Tutorials, offering adaptive holographic guidance tailored to the user’s progress. Our contributions include the development of an XR application with adaptive guidance for intricate tasks, an intuitive user interface evaluated through a user study with 20 participants, and a proof-of-concept for sequence matching methods to assess trainee progress. The study reveals a positive user experience, with the application achieving an average rating of 7.15 out of 10 and a "Good" mark on the System Usability Scale (SUS). Future work aims to expand on the application of Dynamic Time Warping (DTW) for improved task assessment, incorporate critical pose snapshots, and enhance task explanation and item tracking within the app.
Research semester project as part of 3D Vision lecture at ETH Zurich,
Spring 2024
Tor Børve Rasmussen
Faye Zhang
Styrmir Óli Þorsteinsson
Haoran Xu
Video Playlist: https://www.youtube.com/playlist?list=PL9ljg0RPLMToyLbPvqVUjwlWdnGND72LN
See INSTALL file
- MixedReality/com.microsoft.mixedreality.toolkit.examples-2.7.3.tgz
- MixedReality/com.microsoft.mixedreality.toolkit.extensions-2.7.3.tgz
- MixedReality/com.microsoft.mixedreality.toolkit.foundation-2.7.3.tgz
- MixedReality/com.microsoft.mixedreality.toolkit.standardassets-2.7.3.tgz
- MixedReality/com.microsoft.mixedreality.toolkit.testutilities-2.7.3.tgz
- MixedReality/com.microsoft.mixedreality.toolkit.tools-2.7.3.tgz
- All Mixed Reality packages above are available through the Mixed Reality Feature Tool Link
- For a more extensive list of packages used, see here