Skin cancer is the most common cancer in the Western world. In order to accurately detect the disease, especially malignant melanoma-the most fatal form of skin cancer-at an early stage when the prognosis is excellent, there is an urgent need to develop noninvasive early detection methods. We believe that polarization speckle patterns, defined as a spatial distribution of depolarization ratio of traditional speckle patterns, can be an important tool for skin cancer detection. To demonstrate our technique, we conduct a large in vivo clinical study of 214 skin lesions, and show that statistical moments of the polarization speckle pattern could differentiate different types of skin lesions, including three common types of skin cancers, malignant melanoma, squamous cell carcinoma, basal cell carcinoma, and two benign lesions, melanocytic nevus and seborrheic keratoses. In particular, the fourth order moment achieves better or similar sensitivity and specificity than many well-known and accepted optical techniques used to differentiate melanoma and seborrheic keratosis.