This Letter introduces an efficient human detection method in thermal images, using a center-symmetric local binary pattern (CS-LBP) with a luminance saliency map and a random forest (RF) classifier scheme. After detecting a candidate human region, we crop only the head and shoulder region, which has a higher thermal spectrum than the legs or trunk. The CS-LBP feature is then extracted from the luminance saliency map of a hotspot and applied to the RF classifier, which is an ensemble of randomized decision trees. We demonstrate that our detection method is more robust than conventional feature descriptors and classifiers in thermal images.