Due to the widespread use and potential toxicity of avermectins (AVMs), multi-residue monitoring of AVMs in edible tissues, especially in milk, has become increasingly important. With the aim of developing a broad-selective immunoassay for AVMs, a broad-specific monoclonal antibody (Mab) was raised. Based on this Mab, a homologous indirect enzyme-linked immunosorbent assay (ELISA) for the rapid detection of AVMs in milk was developed. Under the optimized conditions, the IC₅₀ values in assay buffer were estimated to be 3.05 ng/mL for abamectin, 13.10 ng/mL for ivermectin, 38.96 ng/mL for eprinomectin, 61.00 ng/mL for doramectin, 14.38 ng/mL for emamectin benzoate. Detection capability (CCβ) of the ELISA was less than 5 ng/mL and 2 ng/mL in milk samples prepared by simple dilution and solvent extraction, respectively. The optimized ELISA was used to quantify AVMs in milk samples spiked at different amounts. The mean recovery and coefficient of variation (CV) were 95.90% and 15.42%, respectively. The Mab-based ELISA achieved a great improvement in AVMs detection. Results proved this broad-selective ELISA would be useful for the multi-residue determination of AVMs in milk without purification process.