We investigated electronic structure of hexagonal multiferroic YMnO3 using the polarization dependent x-ray absorption spectroscopy (XAS) at O K and Mn L(2,3) edges. The spectra exhibit strong polarization dependence at both edges, reflecting anisotropic Mn 3d orbital occupation. Moreover, the O K edge spectra show that Y 4d states are strongly hybridized with O 2p ones, resulting in large anomalies in Born effective charges on off-centering Y and O ions. These results manifest that the Y d(0)-ness with rehybridization is the driving force for the ferroelectricity, and suggest a new approach to understand the multiferroicity in the hexagonal manganites.