The clinical success of inhibitors targeting the PD-1/PD-L1 pathway has made this an active field in cancer immunotherapy. Currently, most drugs targeting this pathway are monoclonal antibodies. Small-molecule inhibitors as the alternative to monoclonal antibodies are expected to overcome the disadvantages of mAbs which include production difficulties and their long half-life. Recently, progress has been reported on anti-PD-1/PD-L1 small-molecule inhibitors. In this paper, we review the development of inhibitors targeting the PD-1/PD-L1 pathway, focusing mainly on peptide-based and nonpeptidic small-molecule inhibitors. The structures and the preclinical and clinical studies of several peptide-based small-molecule candidate compounds in clinical trials are discussed. We also illustrate the design strategies underlying reported nonpeptidic small-molecule inhibitors and provide insight into possible future exploration. Development of small-molecule drugs for anti-PD-1/PD-L1 activity with specific cancer applications is a promising and challenging prospect.