Yttrium oxide (yttria) with monoclinic structure exhibits unique optical properties; however, the monoclinic phase is thermodynamically stable only at pressures higher than ∼16 GPa. In this study, the effect of grain size and plastic strain on the stability of monoclinic phase is investigated by a high-pressure torsion (HPT) method. A cubic-to-monoclinic phase transition occurs at 6 GPa, which is ∼10 GPa below the theoretical transition pressure. Microstructure analysis shows that monoclinic phase forms in nanograins smaller than ∼22 nm and its fraction increases with plastic strain, while larger grains have a cubic structure. The band gap decreases and the photoluminescence features change from electric dipole to mainly magnetic dipole without significant decrease in the photoluminescence intensity after formation of the monoclinic phase. It is also suggested that monoclinic phase formation is due to the enhancement of effective internal pressure in nanograins.