Cancer growth and progression are associated with immune suppression. Cancer cells have the ability to activate different immune checkpoint pathways that harbor immunosuppressive functions. Monoclonal antibodies that target immune checkpoints provided an immense breakthrough in cancer therapeutics. Among the immune checkpoint inhibitors, PD-1/PD-L1 and CTLA-4 inhibitors showed promising therapeutic outcomes, and some have been approved for certain cancer treatments, while others are under clinical trials. Recent reports have shown that patients with various malignancies benefit from immune checkpoint inhibitor treatment. However, mainstream initiation of immune checkpoint therapy to treat cancers is obstructed by the low response rate and immune-related adverse events in some cancer patients. This has given rise to the need for developing sets of biomarkers that predict the response to immune checkpoint blockade and immune-related adverse events. In this review, we discuss different predictive biomarkers for anti-PD-1/PD-L1 and anti-CTLA-4 inhibitors, including immune cells, PD-L1 overexpression, neoantigens, and genetic and epigenetic signatures. Potential approaches for further developing highly reliable predictive biomarkers should facilitate patient selection for and decision-making related to immune checkpoint inhibitor-based therapies.