Self-Supervised Modeling of Activity and Sleep from Wrist Accelerometry – A Generative Approach

Physical activity and sleep are essential to overall health and well-being. However, collecting high-fidelity measurements of fine-grained activities and sleep stages is expensive and cumbersome. Consequently, large epidemiological studies often rely on wrist-worn accelerometers for objective but low-detail activity and sleep metrics. Meanwhile, wearable consumer devices can now provide reasonable estimates of sleep stages, sleep apnea risk, and activity metrics using proprietary algorithms. These devices are becoming increasingly popular, creating a gap as device capabilities outpace epidemiological evidence to guide health recommendations. This highlights the need for robust generalizable accelerometry models to enable more detailed studies of sleep and activity in large cohorts. Development of deep learning models has been limited by the amount of available accelerometry data with activity or sleep stage annotations, and generally failed to improve on hand-crafted feature-based machine learning. Self-supervised learning offers a way to overcome these limitations but is still relatively unexplored for accelerometry. These methods allow deep learning models to be pretrained on large-scale unlabeled datasets.

This thesis focuses on masked autoencoders, a subset of generative self-supervised learning paradigms that enable learning of temporal dependencies in the data during pretraining. This contrasts with previous work which has focused on modeling short windows of accelerometry. Masked autoencoders use reconstruction of masked data segments as a pretraining task. Paper I, introduced a frequency-domain reconstruction objective for pretraining. Resulting downstream model performance on human activity recognition exceeded that of supervised and other self-supervised models. Until recently, wrist accelerometers were considered unable to provide estimates of physiological signals, limiting their utility in sleep staging. However, newer high-sensitivity devices have been found able to detect small wrist movements arising from pulse and respiration. In Paper II, this was leveraged to emphasize these minute pulse or respiration signals in masked reconstructions. A respiration-focused pretraining objective improved sleep staging performance across devices and cohorts, again surpassing other supervised and self-supervised models. It was also shown that the same model supports sleep apnea risk estimation, an important target in overall sleep health evaluation.

To the best of our knowledge, the methods developed in this thesis constitute the first demonstration of the effectiveness of combining domain knowledge with masked sequence modeling in self-supervised pretraining for accelerometry-based activity recognition and sleep staging. The resulting models can provide a foundation for future epidemiological studies of physical activity, sleep, and health.