Individual versus Group Calibration of Machine Learning Models for Physical Activity Assessment Using Body-Worn Accelerometers

Abstract

Purpose: We sought to determine if individually calibrated machine learning models yielded higher accuracy than a group calibration approach for physical activity intensity assessment.

Methods: Participants (n = 48) wore accelerometers on the right hip and nondominant wrist while performing activities of daily living in a semistructured laboratory and/or free-living setting. Criterion measures of activity intensity (sedentary, light, moderate, vigorous) were determined using direct observation. Data were reintegrated into 30-s epochs, and eight random forest models were created to determine physical activity intensity by using all possible conditions of training data (individual vs group), protocol (laboratory vs free-living), and placement (hip vs wrist). A 2 × 2 × 2 repeated-measures analysis of variance was used to compare epoch-level accuracy statistics (% accuracy, kappa [κ]) of the models when used to determine activity intensity in an independent sample of free-living participants.

Results: Main effects were significant for the type of training data (group: accuracy = 80%, κ = 0.59; individual: accuracy = 74% [P = 0.02], κ = 0.50 [P = 0.01]) and protocol (free-living: accuracy = 81%, κ = 0.63; laboratory: accuracy = 74% [P = 0.04], κ = 0.47 [P < 0.01]). Main effects were not significant for placement (hip: accuracy = 79%, κ = 0.58; wrist: accuracy = 75% [P = 0.18]; κ = 0.52 [P = 0.18]). Point estimates for mean absolute error were generally lowest for the group training, free-living protocol, and hip placement.

Conclusions: Contrary to expectations, individually calibrated machine learning models yielded poorer accuracy than a traditional group approach. In addition, models should be developed in free-living settings when possible to optimize predictive accuracy.