Chimeric antigen receptor (CAR) T cell therapy has shown considerable promise for hematologic malignancies, leading to the US Food and Drug Administration approval of two CAR T cell-based therapies for the treatment of B cell acute lymphoblastic leukemia and large B cell lymphoma. Despite success in hematologic malignancies, the treatment landscape of CAR T cell therapy for solid tumors has been limited. There are unique challenges in the development of novel CAR T cell therapies to improve both safety and efficacy. Improved understanding of the immunosuppressive tumor microenvironment and resistance mechanisms has led to encouraging approaches to mitigating these obstacles. This Review will characterize challenges with current CAR T designs for hematologic malignancies and solid tumors and emphasize preclinical and clinical strategies to overcome them with novel CAR T cell therapies.