Teleoperation and Visualization Interfaces for Remote Intervention in Space

Peter Kazanzides; Balazs P Vagvolgyi; Will Pryor; Anton Deguet; Simon Leonard; Louis L Whitcomb

doi:10.3389/frobt.2021.747917

Teleoperation and Visualization Interfaces for Remote Intervention in Space

Front Robot AI. 2021 Dec 1:8:747917. doi: 10.3389/frobt.2021.747917. eCollection 2021.

Authors

Peter Kazanzides^{1

2}, Balazs P Vagvolgyi², Will Pryor^{1

2}, Anton Deguet², Simon Leonard², Louis L Whitcomb^{1

2

3}

Affiliations

¹ Department of Computer Science, Johns Hopkins University, Baltimore, MD, United States.
² Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, United States.
³ Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD, United States.

Abstract

Approaches to robotic manufacturing, assembly, and servicing of in-space assets range from autonomous operation to direct teleoperation, with many forms of semi-autonomous teleoperation in between. Because most approaches require one or more human operators at some level, it is important to explore the control and visualization interfaces available to those operators, taking into account the challenges due to significant telemetry time delay. We consider one motivating application of remote teleoperation, which is ground-based control of a robot on-orbit for satellite servicing. This paper presents a model-based architecture that: 1) improves visualization and situation awareness, 2) enables more effective human/robot interaction and control, and 3) detects task failures based on anomalous sensor feedback. We illustrate elements of the architecture by drawing on 10 years of our research in this area. The paper further reports the results of several multi-user experiments to evaluate the model-based architecture, on ground-based test platforms, for satellite servicing tasks subject to round-trip communication latencies of several seconds. The most significant performance gains were obtained by enhancing the operators' situation awareness via improved visualization and by enabling them to precisely specify intended motion. In contrast, changes to the control interface, including model-mediated control or an immersive 3D environment, often reduced the reported task load but did not significantly improve task performance. Considering the challenges of fully autonomous intervention, we expect that some form of teleoperation will continue to be necessary for robotic in-situ servicing, assembly, and manufacturing tasks for the foreseeable future. We propose that effective teleoperation can be enabled by modeling the remote environment, providing operators with a fused view of the real environment and virtual model, and incorporating interfaces and control strategies that enable interactive planning, precise operation, and prompt detection of errors.

Keywords: model-mediated control; satellite servicing; scene modeling; space robotics; teleoperation.