Despite great enthusiasm towards immunotherapy, reliable biomarkers are still lacking. The importance of subsets based on human papillomavirus (HPV) status is supported by a growing body of evidence. However, role of other possible subgroups such as anatomic localization of primary tumor remains controversial. Our objective was to investigate immune cell infiltrate and checkpoint inhibitor proteins in above-mentioned head and neck cancer subsets. Archival tumor samples of 106 laryngeal, oropharyngeal, and hypopharyngeal cancer patients were stained with PD-L1, PD-L2, PD-1, and CTLA-4 antibodies. Proportion of tumor-infiltrating lymphocytes was assessed as well. In HPV-negative tumors, PD-L1 immune cell positivity was associated with better disease-specific survival. PD-L1 expression on immune cells correlated with improved disease-specific survival in laryngeal tumors. Furthermore, PD-L1 immune cell positivity correlated with CTLA-4 expression on immune cells and it was accompanied by high tumor-infiltrating lymphocyte rate. PD-L1 expression on tumor cells and PD-1 status showed strong correlation in all patients and in oropharyngeal and laryngeal localization, but not in hypopharynx. HPV-negative oropharyngeal cancers showed negative PD-L1 status on tumor cells. CTLA-4 positivity was observed in 49.5% and 20.6% on immune cells and on tumor cells, respectively. We concluded that PD-L1 expression on immune cells indicates better prognosis in laryngeal squamous cell carcinoma and in HPV-negative head and neck squamous cell carcinoma. We have not found any essential differences between anatomic subgroups. A possibly distinct role of hypopharyngeal localization regarding immune activity requires further clarification.