The efficacies of pembrolizumab and nivolumab have never been directly compared in a real-world study. Therefore, we sought to retrospectively evaluate the objective response rate (ORR) and the progression-free survival (PFS) of patients with recurrent or advanced non-small cell lung cancer (NSCLC) in a real-world setting. This study included patients with recurrent or advanced NSCLC diagnosed between September 1, 2015 and August 31, 2019, who were treated with programmed cell death 1 (PD-1) inhibitors at the Cancer Center of the Chinese People's Liberation Army. PFS was estimated for each treatment group using Kaplan-Meier curves and log-rank tests. The multivariate analysis of PFS was performed with Cox proportional hazards regression models. A total of 255 patients with advanced or recurrent NSCLC treated with PD-1 inhibitors were identified. The ORR was significantly higher in the pembrolizumab group than in the nivolumab group, while PFS was not significantly different between the two groups. Subgroup analysis showed that the ORR was significantly higher for pembrolizumab than for nivolumab in patients in the first-line therapy subgroup and in those in the combination therapy as first-line therapy subgroup. Survival analysis of patients receiving combination therapy as second- or further-line therapy showed that nivolumab had better efficacy than pembrolizumab. However, the multivariate analysis revealed no significant difference in PFS between patients treated with pembrolizumab and those treated with nivolumab regardless of the subgroup. In our study, no significant difference in PFS was noted between patients treated with pembrolizumab and those treated with nivolumab in various clinical settings. This supports the current practice of choosing either pembrolizumab or nivolumab based on patient preferences.