Topographic design in wearable MXene sensors with in-sensor machine learning for full-body avatar reconstruction

Haitao Yang; Jiali Li; Xiao Xiao; Jiahao Wang; Yufei Li; Kerui Li; Zhipeng Li; Haochen Yang; Qian Wang; Jie Yang; John S Ho; Po-Len Yeh; Koen Mouthaan; Xiaonan Wang; Sahil Shah; Po-Yen Chen

doi:10.1038/s41467-022-33021-5

Topographic design in wearable MXene sensors with in-sensor machine learning for full-body avatar reconstruction

Nat Commun. 2022 Sep 9;13(1):5311. doi: 10.1038/s41467-022-33021-5.

Authors

Haitao Yang^#¹, Jiali Li^#¹, Xiao Xiao^#², Jiahao Wang^#³, Yufei Li¹, Kerui Li¹, Zhipeng Li³, Haochen Yang⁴, Qian Wang¹, Jie Yang¹, John S Ho³, Po-Len Yeh⁵, Koen Mouthaan³, Xiaonan Wang⁶, Sahil Shah⁷, Po-Yen Chen^{8

9}

Affiliations

¹ Department of Chemical and Biomolecular Engineering, National University of Singapore, 4 Engineering Drive 4, Singapore, 117585, Singapore.
² Department of Electrical and Electronic Engineering, Southern University of Science and Technology, Shenzhen, China.
³ Department of Electrical and Computer Engineering, National University of Singapore, Singapore, 117583, Singapore.
⁴ Department of Chemical and Biomolecular Engineering, University of Maryland, College Park, MD, 20740, USA.
⁵ Realtek, Singapore, 609930, Singapore.
⁶ Department of Chemical Engineering, Tsinghua University, 100084, Beijing, China.
⁷ Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, 20740, USA. sshah389@umd.edu.
⁸ Department of Chemical and Biomolecular Engineering, University of Maryland, College Park, MD, 20740, USA. checp@umd.edu.
⁹ Maryland Robotics Center, College Park, MD, 20740, USA. checp@umd.edu.

^# Contributed equally.

Abstract

Wearable strain sensors that detect joint/muscle strain changes become prevalent at human-machine interfaces for full-body motion monitoring. However, most wearable devices cannot offer customizable opportunities to match the sensor characteristics with specific deformation ranges of joints/muscles, resulting in suboptimal performance. Adequate wearable strain sensor design is highly required to achieve user-designated working windows without sacrificing high sensitivity, accompanied with real-time data processing. Herein, wearable Ti₃C₂T_x MXene sensor modules are fabricated with in-sensor machine learning (ML) models, either functioning via wireless streaming or edge computing, for full-body motion classifications and avatar reconstruction. Through topographic design on piezoresistive nanolayers, the wearable strain sensor modules exhibited ultrahigh sensitivities within the working windows that meet all joint deformation ranges. By integrating the wearable sensors with a ML chip, an edge sensor module is fabricated, enabling in-sensor reconstruction of high-precision avatar animations that mimic continuous full-body motions with an average avatar determination error of 3.5 cm, without additional computing devices.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Accelerometry* / instrumentation
Humans
Machine Learning*
Motion
Range of Motion, Articular*
Wearable Electronic Devices*