Sleep misperception has long been a major issue in the field of insomnia research. Most studies of sleep misperception examine sleep underestimation by comparing the results of polysomnography conducted in a laboratory environment with patients' sleep diary entries. We aimed to investigate psychosocial characteristics of adults who underestimated or overestimated sleep time in a nonclinical, middle-aged community-dwelling population. We collected one week of sleep data with wrist-worn accelerometers. We used egocentric social network analysis to analyze the effects of psychosocial factors. Among 4,060 study participants, 922 completed the accelerometer substudy. Underestimation was defined as an accelerometer-measured sleep time ≥ 6 h and a subjective sleep time < 6 h. Overestimation was defined as an objective sleep time < 6 h and a subjective sleep time ≥ 6 h. Psychosocial characteristics of the sleep misperception group were evaluated using multivariate regression analysis. A total of 47 participants underestimated sleep time, and 420 overestimated sleep time. Regression analysis revealed that women, living with spouse, economic satisfaction, and bridging potential had protective effects against sleep underestimation. Blame from a spouse involved a 3.8-times higher risk of underestimation than the control group (p = 0.002). In men, discussing concerns with a spouse had a protective effect against underestimation (p < 0.001). Economic satisfaction, feeling social network-based intimacy, and support from a spouse were associated with overestimation in women. In men, feeling social network-based intimacy was also associated with overestimation (p < 0.001). We found that social relationship quality was related to sleep overestimation and underestimation. This association was marked in women. Good social relationships may have positive effects on sleep misperception via attenuation of negative emotional reactions and effects on emotional regulation.