Prompt-enhanced hierarchical transformer elevating cardiopulmonary resuscitation instruction via temporal action segmentation

Abstract

The vast majority of people who suffer unexpected cardiac arrest are performed cardiopulmonary resuscitation (CPR) by passersby in a desperate attempt to restore life, but endeavors turn out to be fruitless on account of disqualification. Fortunately, many pieces of research manifest that disciplined training will help to elevate the success rate of resuscitation, which constantly desires a seamless combination of novel techniques to yield further advancement. To this end, we collect a specialized CPR video dataset in which trainees make efforts to behave resuscitation on mannequins independently in adherence to approved guidelines, promoting an auxiliary toolbox to assist supervision and rectification of intermediate potential issues via modern deep learning methodologies. Our research empirically views this problem as a temporal action segmentation (TAS) task in computer vision, which aims to segment an untrimmed video at a frame-wise level. Here, we propose a Prompt-enhanced hierarchical Transformer (PhiTrans) that integrates three indispensable modules, including a textual prompt-based Video Features Extractor (VFE), a transformer-based Action Segmentation Executor (ASE), and a regression-based Prediction Refinement Calibrator (PRC). The backbone preferentially derives from applications in three approved public datasets (GTEA, 50Salads, and Breakfast) collected for TAS tasks, which experimentally facilitates the model excavation on the CPR dataset. In general, we probe into a feasible pipeline that elevates the CPR instruction qualification via action segmentation equipped with novel deep learning techniques. Associated experiments on the CPR dataset advocate our resolution with surpassing 91.0% on Accuracy, Edit score, and F1 score.

Keywords: Boundary regression refinement; Instructional cardiopulmonary resuscitation; Prompt; Temporal action segmentation; Transformer.