Proposal and Evaluation of Visual Haptics for Manipulation of Remote Machine System

Masaki Haruna; Masaki Ogino; Toshiaki Koike-Akino

doi:10.3389/frobt.2020.529040

Proposal and Evaluation of Visual Haptics for Manipulation of Remote Machine System

Front Robot AI. 2020 Oct 7:7:529040. doi: 10.3389/frobt.2020.529040. eCollection 2020.

Authors

Masaki Haruna^{1

2}, Masaki Ogino², Toshiaki Koike-Akino³

Affiliations

¹ Advanced Technology R&D Center, Mitsubishi Electric Corporation (MELCO), Amagasaki, Japan.
² Faculty of Informatics, Kansai University, Takatsuki, Japan.
³ Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA, United States.

Abstract

Remote machine systems have drawn a lot of attention owing to accelerations of virtual reality (VR), augmented reality (AR), and the fifth generation (5G) networks. Despite recent trends of developing autonomous systems, the realization of sophisticated dexterous hand that can fully replace human hands is considered to be decades away. It is also extremely difficult to reproduce the sensilla of complex human hands. On the other hand, it is known that humans can perceive haptic information from visual information even without any physical feedback as cross modal sensation between visual and haptics sensations or pseudo haptics. In this paper, we propose a visual haptic technology, where haptic information is visualized in more perceptual images overlaid at the contact points of a remote machine hand. The usability of the proposed visual haptics was evaluated by subject's brain waves aiming to find out a new approach for quantifying "sense of oneness." In our proof-of-concept experiments using VR, subjects are asked to operate a virtual arm and hand presented in the VR space, and the performance of the operation with and without visual haptics information as measured with brain wave sensing. Consequently, three results were verified. Firstly, the information flow in the brain were significantly reduced with the proposed visual haptics for the whole α, β, and θ-waves by 45% across nine subjects. This result suggests that superimposing visual effects may be able to reduce the cognitive burden on the operator during the manipulation for the remote machine system. Secondly, high correlation (Pearson correlation factor of 0.795 at a p-value of 0.011) was verified between the subjective usability points and the brainwave measurement results. Finally, the number of the task successes across sessions were improved in the presence of overlaid visual stimulus. It implies that the visual haptics image could also facilitate operators' pre-training to get skillful at manipulating the remote machine interface more quickly.

Keywords: EEG; brain wave; manipulation; remote machine system; visual haptics.