Sleep is the most important period for recovering from daily stress and load. Assessment of the stress recovery during sleep is therefore, an important metric for care and quality of life. Heart rate variability (HRV) is a non-invasive marker of autonomic nervous system (ANS) activity, and HRV-based methods can be used to assess physiological recovery, characterized by parasympathetic domination of the ANS. HRV is affected by multiple factors of which some are unmodifiable (such as age and gender) but many are related to daily lifestyle choices (e.g. alcohol consumption, physical activity, sleeping times). The purpose of this study was to investigate the association of these aforementioned factors on HRV-based recovery during sleep on a large sample. Variable importance measures yielded by random forest were used for identifying the most relevant predictors of sleep-time recovery. The results emphasize the disturbing effects of alcohol consumption on sleep-time recovery. Good physical fitness is associated to good recovery, but acute physical activity seems to challenge or delay the recovery process for the next night. Longer sleeping time enables more recovery minutes, but the proportion of recovery (i.e. recovery efficiency) seems to peak around 7.0-7.25 hours of sleep.