Mixed Reality Human-Robot Interface for Robotic Operations in Hazardous Environments

Abstract

This thesis addresses the need for reliable and immersive human-robot interfaces in hazardous environments. Three key goals were achieved. Firstly, techniques were developed to provide immersive interactions while ensuring accurate robot positioning in a realistic underground particle accelerator environment. A network optimization framework facilitated using Mixed Reality (MR) technologies for collision detection, trajectory planning, and real-time control. Secondly, an Augmented Reality (AR) interface was designed for wireless network environments, enabling efficient interaction through hand and eye tracking, motion tracking, and voice recognition. The interface allowed collaboration among multiple experts in the AR workspace. The interface's performance was evaluated in real intervention scenarios at CERN, meeting operational requirements. A comparison with a 2D interface was conducted, assessing safety and efficiency using various assessment methods and physiological measurements. The developed MR human-robot interfaces demonstrated operational readiness. Limitations and areas for improvement were identified, including optimizing network architecture and applying automatic interaction strategies.