The efficacy of immunotherapy is largely patient-specific due to heterogeneity in tumors. Combining statistic power from a variety of immunotherapies across cancer types, we found four biological pathways significantly correlated with patient survival following immunotherapy. The expression of immunotherapy prognostic marker genes (IPMGs) in these pathways can predict the patient survival with high accuracy not only in the TCGA cohort (89.36%) but also in two other independent cohorts (80.91%), highlighting that the activity of the IPMGs can reflect the sensitivity of the tumor immune microenvironment (TIME) to immunotherapies. Using mouse models, we show that knockout of one of the IPMGs, MALT1, which is critical for the T-cell receptor signaling, can eliminate the antitumor effect of anti-PD-1 treatment completely by impairing the activation of CD8+ T cells. Notably, knockout of another IPMG, CLEC4D, a C-type lectin receptor that expressed on myeloid cells, also reduced the effect of anti-PD-1 treatment potentially through maintaining the immunosuppressive effects of myeloid cells. Our results suggest that priming TIME via activating the IPMGs may increase the response rate and the effect of immune checkpoint blockers.