The COVID-19 pandemic might affect many aspects of the community and a range of psychiatric risk factors due to life changes, including people's behaviors and perceptions. In this study, we aim to identify specific life changes that correlate with psychological distress within the social context of the COVID-19 pandemic in Japan. In July 2020, workers (company employees and civil servants) in Japan were recruited from local institutions that had not had any confirmed COVID-19 cases as well as neighborhoods that had only a few cases. Participants completed a COVID-19 mental health survey (N = 609; 66.9% male). Psychological distress was identified based on Kessler-6 scores (≥13). Life changes were assessed by an open-ended question about life changes in participants and their family, workplace, and community due to the COVID-19 pandemic. A convergent mixed-method approach was used to compare the context of perceived life changes in participants with psychological distress and those without. As a result, 8.9% of participants had psychological distress, and sex and age categories were different between those with psychological distress and those without. Among the participants who responded to the open-ended question, the biggest life change was "staying at home," and the next biggest life changes were "event cancellations" and "increased workload" in participants with psychological distress, and "no changes" and "mask-wearing" in those without psychological distress, respectively. Regarding emotional/perceptual changes, "stress," "fear," and "anger" were more frequently reported by participants with psychological distress than those without (P <0.001). By integrating these findings, we identified themes focusing on vulnerable characteristics related to psychological distress. This study may provide a source in society for mediating psychological distress during a pandemic.