Kesterite Cu2ZnSnS4 (CZTS), used for thin film solar cells, has a band gap energy around 1.5-1.6 eV with possibilities for further increase through alloying. In some applications for wide band gap solar cells, reduced absorber thickness can be beneficial, to allow partial light transmission. Reduced thickness can also be beneficial to reduce bulk recombination, and so called ultrathin solar cells (<700 nm thick) have been studied for several materials systems. Here, we report performance for CZTS devices down to 250 nm thickness and show that performance loss from thickness reduction is relatively small, partly due to short minority carrier diffusion length. Insertion of thin passivation layers (Al2O3, SiO2 or HfO2) at the Mo/CZTS interface gives improved performance of ultrathin devices, from 4.7% to 5.6% efficiency for best performing cells having 250 nm thick CZTS with Mo as compared to Mo/Al2O3 back contact. The approach of NaF post deposition for making isolating passivation layers conductive is tested for the first time for CZTS and is shown to work. For fabrication of CZTS devices on transparent ITO back contact, the insertion of passivation layers can reduce diffusion of indium into CZTS, but device performance is lower than on Mo back contacts.