Optical conductivity spectra sigma(omega) were used to investigate the effect of orbital ordering on the electronic structure of Ca2RuO4. Our LDA+U calculation predicts Ru 4d(xy) ferro-orbital ordering at the ground state, and well explains the present sigma(omega) as well as the reported O 1s x-ray absorption spectra. Variation of temperature (T) causes a large change of spectral weight over several eV as well as collapse of a charge gap accompanied by elongation of the c-axis Ru-O bond length. These results clearly indicate that the d(xy) orbital ordering plays a crucial role in the metal-insulator transition and the T-dependent electronic structure on a large energy scale.