Autophagy inhibition is an attractive target for cancer therapy. In this study, we discovered inhibitors of Atg4B essential for autophagosome formation and evaluated their potential as therapeutics for prostate cancer. Seventeen compounds were identified as candidates after in silico screening and a thermal shift assay. Among them, compound 17 showed the most potent Atg4B inhibitory activity, inhibited autophagy induced by anti-castration-resistant prostate cancer (CRPC) drugs, and significantly enhanced apoptosis. Although 17 has been known as a phospholipase A2 (PLA2) inhibitor, other PLA2 inhibitors had no effect on Atg4B and autophagy. We then performed structural optimization based on molecular modeling and succeeded in developing 21f (by shortening the alkyl chain of 17), which was a potent competitive inhibitor for Atg4B (Ki = 3.1 μM) with declining PLA2 inhibitory potency. Compound 21f enhanced the anticancer activity of anti-CRPC drugs via autophagy inhibition. These findings suggest that 21f can be used as an adjuvant drug for therapy with anti-CRPC drugs.