Artifacts such as hair are major obstacles to automatic segmentation of pigmented skin lesion images for computer-aided diagnosis systems. It is even more challenging to process clinical images taken by a regular digital camera, where the shadows of the skin texture may mimic hair-like curvilinear structures. In this study, we examined the popular DullRazor software with a dataset of 20 clinical images. The software, specifically designed for dermoscopic images, was unable to remove fine hairs or hairs in the shade. Alternatively, we proposed using conventional matched filters to enhance curvilinear structures. The more complicate hair intersection patterns, which were known to generate low matched filtering responses, were recovered by using region growing algorithms from nearby detected hair segments with linear discriminant analysis (LDA) based on a color similarity criterion. The preliminary results indicated the proposed method was able to remove more fine hairs and hairs in the shade, and lower false hair detection rate by 58% (from 0.438 to 0.183) as compared to the DullRazor's approach.