The band gap and defect features of ultrathin ZrO2 films with varying thicknesses have been investigated by spectroscopic ellipsometry through the point-by-point data inversion method. The ε2-sprectra in the 3-6 eV range are extracted based on an optical model consisting of a Si substrate/effective ZrO2 film/air ambient structure where the effective ZrO2 film is a combination of interfacial layers and ZrO2. Evident widening of the band gap with a reducing size is observed when the effective ZrO2 films are below a critical thickness, somewhere between 8.80 nm and 17.13 nm. This is due to quantum-confinement and amorphous effects. Moreover, the sub-band-gap defects at interfacial layers and in bulk ZrO2 are identified and present strong thickness dependence as well. The interfacial defects at 3.26, 4.13, 4.43, and 4.77 eV mainly exist below the critical thickness and exhibit a significant suppression with increasing film thickness. The bulk defects at 4.15 eV and 4.46 eV dominate in ZrO2 films once they are over the critical thickness. The evolution of the band gap and defects is closely related to variance in the electronic structure of amorphous ZrO2. Our results may be helpful in understanding controversial problems concerning the size effect on ultrathin high-k oxide films and exploring the further miniaturization of electronic devices based on them.