Due to the unique antibacterial properties, silver nanoparticles (AgNPs) have been widely used in commercial applications. In this study, the toxicity of three kinds of AgNPs with different sizes and surface coatings to marine diatom Skeletonema costatum (S. costatum) was studied, which was one of the dominant species in estuarine and coastal areas. All three kinds of tested AgNPs inhibited the growth of exposed S. costatum under acute exposure condition, and the order of toxicity was 10 nm-OA > 10 nm-PVP > 20 nm-PVP. Given the condition of similar particle size, oil amine surface coated AgNPs were more toxic than polyvinyl pyrrolidone (PVP) surface coated AgNPs in S. costatum in term of cytotoxicity. With the same surface coating, the toxicity of AgNPs in S. costatum was affected by its hydrodynamic diameter and exposure concentrations. When the concentration of AgNPs was less than 500 µg · L⁻¹, larger sized AgNPs showed greater toxicity; When the concentration was greater than or equal to 500 µg · L⁻¹, smaller AgNPs exhibited greater toxicity. At molecular level, 50 µg · L⁻¹ 10nm-PVP significantly upregulated expression level of 3HfcpA (P < 0.05) and significantly downregulated expression level of Dl (P < 0.05), and 500 µg · L⁻¹ 10nm-OA significantly upregulated 3HfcpA expression (P < 0.05), while 20 nm-PVP treatment group didn't show any significant change. Exposed diatom demonstrated sensitive photosynthesis response to small size and PVP coated silver nanoparticles at molecular level. This study suggested that the toxicity of AgNPs to marine microalgae was largely controlled by the particle size, surface coating, exposure medium, exposure concentration and other factors. The smaller the particle size, the greater the toxicity of AgNPs, and the particle size of AgNPs played an important role in the toxicity of AgNPs in marine diatom S. costatum.