Background It is uncertain whether a deep learning-based automatic detection algorithm (DLAD) for identifying malignant nodules on chest radiographs will help diagnose lung cancers. Purpose To evaluate the efficacy of using a DLAD in observer performance for the detection of lung cancers on chest radiographs. Materials and Methods Among patients diagnosed with lung cancers between January 2010 and December 2014, 117 patients (median age, 69 years; interquartile range [IQR], 64-74 years; 57 women) were retrospectively identified in whom lung cancers were visible on previous chest radiographs. For the healthy control group, 234 patients (median age, 58 years; IQR, 48-68 years; 123 women) with normal chest radiographs were randomly selected. Nine observers reviewed each chest radiograph, with and without a DLAD. They detected potential lung cancers and determined whether they would recommend chest CT for follow-up. Observer performance was compared with use of the area under the alternative free-response receiver operating characteristic curve (AUC), sensitivity, and rates of chest CT recommendation. Results In total, 105 of the 117 patients had lung cancers that were overlooked on their original radiographs. The average AUC for all observers significantly rose from 0.67 (95% confidence interval [CI]: 0.62, 0.72) without a DLAD to 0.76 (95% CI: 0.71, 0.81) with a DLAD (P < .001). With a DLAD, observers detected more overlooked lung cancers (average sensitivity, 53% [56 of 105 patients] with a DLAD vs 40% [42 of 105 patients] without a DLAD) (P < .001) and recommended chest CT for more patients (62% [66 of 105 patients] with a DLAD vs 47% [49 of 105 patients] without a DLAD) (P < .001). In the healthy control group, no difference existed in the rate of chest CT recommendation (10% [23 of 234 patients] without a DLAD and 8% [20 of 234 patients] with a DLAD) (P = .13). Conclusion Using a deep learning-based automatic detection algorithm may help observers reduce the number of overlooked lung cancers on chest radiographs, without a proportional increase in the number of follow-up chest CT examinations. © RSNA, 2020 Online supplemental material is available for this article.