ROS1 rearrangement is a predictive biomarker for response to the tyrosine kinase inhibitor, crizotinib. We investigated the usefulness of ROS1 immunohistochemistry (IHC) for the detection of patients who harbor ROS1 rearrangements in two separate cohorts. We also compared ROS1 IHC with ALK IHC in terms of diagnostic performance to predict each gene rearrangement. In a retrospective cohort, IHC was performed in 219 cases of lung adenocarcinoma with already known genetic alterations. In a prospective cohort, we performed IHC for 111 consecutive cases of lung adenocarcinoma and confirmed the results by subsequent FISH. In the retrospective cohort, all 8 ROS1-rearranged tumors were immunoreactive, and 14 of 211 ROS1-wild cases were immunoreactive (sensitivity 100% and specificity 93.4%). In the prospective cohort, all IHC-negative cases were FISH-negative, and 5 of 34 ROS1 immunoreactive cases were ROS1-rearranged (sensitivity 100% and specificity 72.6%). In ROS1-wild tumors, ROS1 protein was more expressed in the tumors of ever-smokers than in those of never-smokers (p = 0.003). ALK IHC showed 100% sensitivity and 98.1 to 100% specificity in both patient cohorts. In conclusion, ROS1 IHC is highly sensitive, but less specific compared with ALK IHC for detection of the corresponding rearrangement. ROS1 IHC-reactive tumors, especially when the tumor is stained with moderate to strong intensity or a diffuse pattern, are recommended to undergo FISH to confirm the gene rearrangement.