Human Performance and Cybersickness Evaluation of Mixed Reality Device for Immersive Assembly Simulation Training: A Case Study of Microsoft Hololens 2

Reza Aulia Akbar, Chiuhsiang Joe Lin, Retno Widyaningrum, Adithya Sudiarno, Maria Anityasari

Abstract


A Mixed Reality device is a new technology that provides immersive training experiences. One of the mixed reality technology with high immersion, high interaction, and high information is Microsoft Hololens 2. Smart glasses are classified as OST-HMDs (Optical See-Through Head-Mounted Displays), which display augmented objects in a mixed-reality environment. The use of HMDs can cause cybersickness symptoms that users feel. This study aims to assess the human performance and cybersickness of Hololens 2 for immersive training. The training object was an augmented assembly engine comprising six parts. Using Hololens 2 as an assembly simulator training, participants experienced a significant increase in performance and learning rate with minor errors. Hololens 2 produces light symptoms that the user perceives as general discomfort, fatigue, difficulty focusing, sweating, difficulty concentrating, and blurred vision. However, Hololens 2 experienced negligible symptoms based on the cybersickness factor: Nausea (3.98), Oculomotor (6.32), and Disorientation (2.32). Based on the SSQ scoring matrix assessment, Hololens 2 obtained an SSQ score of 4.48, which is classified as a minor symptom of cybersickness. 

A Mixed Reality device is a new technology that provide immersive training experiences. One of the mixed reality technology with high immersion, high interaction, and high information is Microsoft Hololens 2. Smart glasses are classified as OST-HMDs (Optical See-Through Head-Mounted Displays), which display augmented objects in a mixed-reality environment. The use of HMDs can cause cybersickness symptoms that users feel. This study aims to assess the human performance and cybersickness of Hololens 2 for immersive training. The training object was an augmented assembly engine comprising six parts. Using Hololens 2 as an assembly simulator training, participants experienced a significant increase in performance and learning rate with minor errors. Hololens 2 produces light symptoms that the user perceives: General discomfort, fatigue, difficulty focusing, sweating, difficulty concentrating, and blurred vision. However, Hololens 2 experienced negligible symptoms based on the cybersickness factor: Nausea (3.98), Oculomotor (6.32), and Disorientation (2.32). Based on the SSQ scoring matrix assessment, Hololens 2 obtained an SSQ score of 4.48, which is classified as a minor symptom of cybersickness.


Keywords


Assembly Simulation Training; Cybersickness Evaluation; Human Performance; Immersive Training; Mixed Reality Device

Full Text:

PDF

References


J. Zhu, S. K. Ong and A. Y. C. Nee, "A Context-Aware Augmented Reality System to Assist the Maintenance Operators," International Journal on Interactive Design and Manufacturing, vol. 8, no. 4, pp. 293-304, 2014.

M. Agrawala, D. Phan, J. Heiser, J. Haymaker, J. Klingner, P. Hanrahan and B. Tversky, "Designing Effective Step-by-Step Assembly Instructions," ACM Transactions on Graphics, vol. 22, no. 3, pp. 828-837, 2003.

S. Webel, U. Bockholt, T. Engelke, N. Gavish, M. Olbrich and C. Preusche, "An Augmented Reality Training Platform for Assembly and Maintenance Skills," Robotics and Autonomous Systems, vol. 61, no. 4, pp. 398-403, 2013.

J. Forlizzi and K. Battarbee, "Understanding Experience in Interactive Systems," in Conference on Designing interactive Systems: Processes, Practices, Methods, and Techniques, New York, 2004.

C. Flavián, S. I. Sánchez and C. Orús, "The Impact of Virtual, Augmented and Mixed Reality Technologies on the Customer Experience," Journal of Business Research, vol. 100, pp. 547-560, 2019.

B. J. Park, S. J. Hunt, C. Martin, G. J. Nadolski, B. J. Wood and T. P. Gade, "Augmented and Mixed Reality: Technologies for Enhancing the Future of IR," Journal of Vascular and Interventional Radiology, vol. 31, no. 7, pp. 1074-1082, 2020.

M. Parveau and M. Adda, "3iVClass: a new classification method for Virtual, Augmented and Mixed Realities," in The 9th International Conference on Emerging Ubiquitous Systems and Pervasive Networks (EUSPN), Leuven, 2018.

I. S. Potemin, A. Zhdanov, N. Bogdanov, D. Zhdanov, I. Livshits and Y. Wang, "Analysis of the Visual Perception Conflicts in Designing Mixed Reality Systems," in Proceedings of the Optical Design and Testing VIII, Beijing, 2018.

M. Claydon, "Alternative Realities: From Augmented Reality to Mobile Mixed Reality.," University of Tampere, Tampere, 2015.

L. Qian, A. Barthel, A. Johnson, G. Osgood, P. Kazanzides, N. Navab and B. Fuerst, "Comparison of Optical See-Through Head-Mounted Displays for Surgical Interventions with Object-Anchored 2D-Display," International Journal of Computer Assisted Radiology and Surgery, vol. 12, pp. 901-910, 2017.

J. T. Gibby, S. A. Swenson, S. Cvetko, R. Rao and R. Javan, "Head-Mounted Display Augmented Reality to Guide Pedicle Screw Placement Utilizing Computed Tomography," International Journal of Computer Assisted Radiology and Surgery, vol. 14, pp. 525-535, 2019.

Y. Liu, H. Dong, L. Zhang and A. E. Saddik, "Technical Evaluation of HoloLens for Multimedia: A First Look," IEEE Multimedia, vol. 25, no. 4, pp. 8-18, 2018.

Microsoft Corporation, "Microsoft HoloLens 2," Microsoft, 2022. [Online]. Available: https://www.microsoft.com/en-us/hololens. [Accessed 17 May 2022].

S. Bouchard, M. Berthiaume, G. Robillard, H. Forget, C. D. Peltier, P. Renaud, C. Blais and D. Fiset, "Arguing in Favor of Revising the Simulator Sickness Questionnaire Factor Structure when Assessing Side Effects Induced by Immersions in Virtual Reality," Frontier Psychiatry, vol. 12, no. 739742, pp. 1-11, 2021.

A. Vovk, F. Wild, W. Guest and T. Kuula, "Simulator Sickness in Augmented Reality Training Using the Microsoft HoloLens," in CHI Conference on Human Factors in Computing Systems, Montreal, 2018.

C. L. Hughes, C. Fidopiastis, K. M. Stanney, P. S. Bailey and E. Ruiz, "The Psychometrics of Cybersickness in Augmented Reality," Frontier in Virtual Reality, vol. 1, no. 602954, pp. 1-12, 2020.

R. S. Kennedy, N. E. Lane, K. S. Berbaum and M. G. Lilienthal, "Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness," The International Journal of Aviation Psychology, vol. 3, no. 3, p. 203–220, 1993.

V. Sevinc and M. I. Berkman, "Psychometric Evaluation of Simulator Sickness Questionnaire and Its Variants as a Measure of Cybersickness in Consumer Virtual Environments," Applied Ergonomics, vol. 82, no. 102958, pp. 1-12, 2020.

Dersin Studio, Toyota 4A-GE 1.6L Redtop Engine 3D Model, RenderHub, 2019.

A. D. Patel, A. G. Gallagher, W. J. Nicholson and C. U. Cates, "Learning Curves and Reliability Measures for Virtual Reality Simulation in the Performance Assessment of Carotid Angiography," Journal of the American College of Cardiology, vol. 47, no. 9, pp. 1796-1802, 2006.

R. S. Kennedy, J. Drexler and R. C. Kennedy, "Research in Visually Induced Motion Sickness," Applied Ergonomics, vol. 41, no. 4, pp. 494-503, 2010.

H. Walter, R. Li, J. Munafo, C. Curry, N. Peterson and T. Stoffregen, "A Brief Explanation of the Simulator Sickness Questionnaire (SSQ)," APAL Coupling Study, 2019.

K. M. Stanney, R. S. Kennedy and J. M. Drexler, "Cybersickness is Not Simulator Sickness," 1997.

L. Lauer, K. Altmeyer, S. Malone, M. Barz, R. Brünken, D. Sonntag and M. Peschel, "Investigating the Usability of a Head-Mounted Display Augmented Reality Device in Elementary School Children," Sensors, vol. 21, no. 19, pp. 1-20, 2021.

E. Bottani, F. Longo, L. Nicoletti, A. Padovano, G. P. C. Tancredi, L. Tebaldi, M. Vetrano and G. Vignali, "Wearable and Interactive Mixed Reality Solutions for Fault Diagnosis and Assistance in Manufacturing Systems: Implementation and Testing in an Aseptic Bottling Line," Computers in Industry, vol. 128, pp. 1-14, 2021.




DOI: http://dx.doi.org/10.12962/j25807471.v9i1.22480

Creative Commons License
JMES The International Journal of Mechanical Engineering and Sciences by Lembaga Penelitian dan Pengabdian kepada Masyarakat (LPPM) ITS is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at https://iptek.its.ac.id/index.php/jmes.