Robust and automatic motion-capture data recovery using soft skeleton constraints and model averaging

PLoS One. 2018 Jul 10;13(7):e0199744. doi: 10.1371/journal.pone.0199744. eCollection 2018.

Abstract

Motion capture allows accurate recording of human motion, with applications in many fields, including entertainment, medicine, sports science and human computer interaction. A common difficulty with this technology is the occurrence of missing data, due to occlusions, or recording conditions. Various models have been proposed to estimate missing data. Some are based on interpolation, low-rank properties or inter-correlations. Others involve dataset matching or skeleton constraints. While the latter have the advantage of promoting a realistic motion estimation, they require prior knowledge of skeleton constraints, or the availability of a prerecorded dataset. In this article, we propose a probabilistic averaging method of several recovery models (referred to as Probabilistic Model Averaging (PMA) in this paper), based on the likelihoods of the distances between body points. This method has the advantage of being automatic, while allowing an efficient gap data recovery. To support and validate the proposed method, we use a set of four individual recovery models, based on linear/nonlinear regression in local coordinate systems. Finally, we propose two heuristic algorithms to enforce skeleton constraints in the reconstructed motion, which can be used on any individual recovery model. For validation purposes, random gaps were introduced into motion-capture sequences, and the effects of factors such as the number of simultaneous gaps, gap length and sequence duration were analyzed. Results show that the proposed probabilistic averaging method yields better recovery than (i) each of the four individual models and (ii) two recent state-of-the-art models, regardless of gap length, sequence duration and number of simultaneous gaps. Moreover, both of our heuristic skeleton-constraint algorithms significantly improve the recovery for 7 out of 8 tested motion-capture sequences (p < 0.05), for 10 simultaneous gaps of 5 seconds. The code is available for free download at: https://github.com/numediart/MocapRecovery.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Biomechanical Phenomena
  • Bone and Bones / anatomy & histology
  • Bone and Bones / physiology*
  • Data Accuracy
  • Humans
  • Motion*
  • Movement
  • Software*
  • Video Recording / methods*

Grants and funding

Mickaël Tits is funded through a PHD grant from the Fonds pour la Formation à la Recherche dans l’Industrie et l’Agriculture (FRIA, FRS-FNRS, FC 005499), Belgium, http://www.fnrs.be/en/. The Qualisys motion capture equipment was funded by the Fonds de la Recherche Scientifique (FRS-FNRS, EQP U.N041.13), Belgium. This research project is supported by the European Regional Development Fund (ETR 1212 0000 3303). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.