A series of new UV-curable hybrid coatings were prepared with hyperbranched polyurethane acrylate-polyurethane diacrylate (HBPUA-PUDA) and organic SiO2 as formation, and 2-hydroxyl-2-methyl-l-phenyl-1-propanone (Darocur 1173) was used as photo-initiator. The effect of organic SiO2 content on the thermal stability and conversion of C=C bonds of the UV-cured HBPUA-PUDA/SiO2 films were evaluated using thermal gravimetric analysis (TGA) and Fourier transform infrared spectroscopy (FTIR). The morphology of SiO2 were observed by using transmission electron microscopy (TEM), field emission scanning electron microscopy (FESEM), and the elemental composition of SiO2 was tested by energy dispersive spectrometer (EDS). The results revealed that when organic SiO2 content was less than 30 wt%, the average particle size of SiO2 was about 95 nm and the particles were well-dispersed in the organic phase. The mechanical properties, abrasion resistance and transmittance of the films were also investigated. The results showed that with the increase of organic SiO2 content, the adhesive force and pendulum hardness were increased obviously, while the transmittance and flexibility were decreased. When the content of the organic SiO2 was 30 wt%, the film of HBPUA-PUDA/SiO2 showed the optimal comprehensive performance as follows: pendulum hardness was 0.74, transmittance was 93.1%, adhesion was 0 grade, flexibility was 2 mm, impact strength was 43 Kg x cm and mass loss of 300 cycles was 12.9 mg.