Adaptive attention-based human machine interface system for teleoperation of industrial vehicle

Jouh Yeong Chew; Mitsuru Kawamoto; Takashi Okuma; Eiichi Yoshida; Norihiko Kato

doi:10.1038/s41598-021-96682-0

Adaptive attention-based human machine interface system for teleoperation of industrial vehicle

Sci Rep. 2021 Aug 26;11(1):17284. doi: 10.1038/s41598-021-96682-0.

Authors

Jouh Yeong Chew¹, Mitsuru Kawamoto², Takashi Okuma², Eiichi Yoshida², Norihiko Kato²

Affiliations

¹ National Institute of Advanced Industrial Science and Technology, AIST Tsukuba Central 1, 1-1-1 Umezono, Tsukuba, Ibaraki, 305-8560, Japan. jy.chew@aist.go.jp.
² National Institute of Advanced Industrial Science and Technology, AIST Tsukuba Central 1, 1-1-1 Umezono, Tsukuba, Ibaraki, 305-8560, Japan.

Abstract

This study proposes a Human Machine Interface (HMI) system with adaptive visual stimuli to facilitate teleoperation of industrial vehicles such as forklifts. The proposed system estimates the context/work state during teleoperation and presents the optimal visual stimuli on the display of HMI. Such adaptability is supported by behavioral models which are developed from behavioral data of conventional/manned forklift operation. The proposed system consists of two models, i.e., gaze attention and work state transition models which are defined by gaze fixations and operation pattern of operators, respectively. In short, the proposed system estimates and shows the optimal visual stimuli on the display of HMI based on temporal operation pattern. The usability of teleoperation system is evaluated by comparing the perceived workload elicited by different types of HMI. The results suggest the adaptive attention-based HMI system outperforms the non-adaptive HMI, where the perceived workload is consistently lower as responded by different categories of forklift operators.