Vascular smooth muscle cells (VSMCs) have two distinct phenotypes: contractile and synthetic. The major difference between these phenotypes lies in the magnitude of the contractile force produced by the cell. Although traction force microscopy (TFM) is often used to evaluate cellular contractile force, this method requires complex preprocessing and a sufficiently compliant substrate. To evaluate the contractile force and the phenotype of living VSMCs with minimal effort and in a manner independent of the substrate stiffness, we propose a photoelasticity-based method using retardation, which is related to the difference between the first and second principal stresses and their orientation. The results demonstrate that actin filaments co-localize with areas of high retardation in cells, indicating that the retardation of VSMCs is promoted by actin filaments. The retardation of cells treated with calyculin A and Y-27632 tended to be larger and smaller, respectively, than that of control cells. Cell traction force significantly correlates with total cell retardation (r2 = 0.38). The retardation of contractile VSMCs (passage 2) was significantly higher than that of synthetic VSMCs (passage 12). These results indicate that cell retardation can be used to assess cell contractile force and, thus, determine the phenotype of VSMCs.