The quality monitoring of fiber-optic coil (FOC) in winding systems is usually done manually. Aiming at the problem of inefficient and low accuracy of manual detection, this article is dedicated to researching a defect detection framework based on machine vision, which provides a reliable method for automatic defect detection of FOC. For this purpose, a defect detection scheme that integrates wavelet transform and nonlocal means filtering is proposed to accurately locate the defect region. Then, based on the features constructed by wavelet coefficients, a support vector machine (SVM) is used as the classifier. Additionally, a self-adaptive genetic algorithm is proposed to optimize the parameters of the SVM to form the final classifier. Through experiments on the data set obtained by our designed imaging system, the results show that our method has good defect detection performance and high classification accuracy, which provides an optimal solution for the automatic detection of FOC.