Emory Meursing
ABSTRACT
With the rapidly growing virtual reality (VR) technologies, maturing motion tracking applications, and the advancement in robotics, physically interacting with the remote world via a robot while the user is observing the scene through a VR headset seems to be one solution to the substantial need for telepresence. This work presents a first-person viewing hand motion controlled robotic arm framework that provides an immersive robot control experience. In particular, we developed a platform with an off-the-shelf 360° camera, a robotic arm, and a VR headset to allow for live 360° video streaming, hand tracking, and robotic arm control. Demo activities with three types of robot operation tasks (i.e., in-air drawing, object collection, and line tracing) show that our platform enables an enhanced first-person vision presence and a smooth robot teleoperation.
- Meta. 2021. What is Virtual Reality All About?https://www.oculus.com/blog/what-is-virtual-reality-all-about/Google Scholar
- Meta. 2022. META QUEST 2. https://store.facebook.com/quest/products/quest-2Google Scholar
- Pollen Robotics. 2022. Reachy can now move around!https://www.pollen-robotics.com/Google Scholar
- Ryo Suzuki, Adnan Karim, Tian Xia, Hooman Hedayati, and Nicolai Marquardt. 2022. Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces. In CHI Conference on Human Factors in Computing Systems. 1–33.Google ScholarDigital Library
- Ievgenii A Tsokalo, David Kuss, Ievgen Kharabet, Frank HP Fitzek, and Martin Reisslein. 2019. Remote robot control with human-in-the-loop over long distances using digital twins. In 2019 IEEE Global Communications Conference (GLOBECOM). IEEE, 1–6.Google ScholarDigital Library
- Ultrleap. 2022. Leap Motion Controller. https://www.ultraleap.com/product/leap-motion-controller/Google Scholar
Index Terms
- Demo: I Am a Robot: First-person Robotic Arm Control with Hand Motions in Virtual Reality
Recommendations
Virtual, Augmented, and Mixed Reality for Human-Robot Interaction
HRI '18: Companion of the 2018 ACM/IEEE International Conference on Human-Robot InteractionThe 1st International Workshop on Virtual, Augmented, and Mixed Reality for Human-Robot Interactions (VAM-HRI) will bring together HRI, Robotics, Artificial Intelligence, and Mixed Reality researchers to identify challenges in mixed reality interactions ...
Exploiting virtual reality and the robot operating system to remote-control a humanoid robot
AbstractThe availability of frameworks and applications in the robotic domain fostered in the last years a spread in the adoption of robots in daily life activities. Many of these activities include the robot teleoperation, i.e. controlling its movements ...
Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (VAM-HRI)
HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot InteractionThe 4th International Workshop on Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (VAM-HRI) will bring together HRI, robotics, and mixed reality researchers to address challenges in mixed reality interactions between humans and robots. ...
Comments