Y2O3:Tb3+ and Y2O3:Tb3+, Yb3+ samples were prepared by co-precipitation method. The morphology, microstructure and fluorescence spectra at room temperature of samples were characterized by scanning electron microscopy (SEM), transmission electron microscopy (TEM), X-ray diffraction (XRD) and fluorescence spectrometer, The optimal process conditions of Y2O3:Tb3+ under different doping concentrations, annealing temperature, and pH value of the solution were obtained: Tb3+ concentration is 1.5%, annealing temperature is 1400 degrees C, an alkaline solution environment, and samples under 300 nm light excitation have the largest green light emission at 543 nm. The corresponding relation of Tb3+ ion level structure and transition properties and experimental spectra were analyzed in detail, and we explained the influence mechanism of process conditions and the fluorescence quenching process mainly effects luminous intensity of samples. The energy transfer from sensitizing ions Tb3+ to active ion Yb3+ was confirmed, it made the sample have considerable emitting light in the near-infrared region; the authors described the process of cooperation conversion luminescence between the two ions from the level transition angle, and also analyzed the system of fluorescence quenching process. Test results showed that the near infrared quantum cutting can effectively improve the luminous efficiency of doped ions, and will have broad application prospects in the silicon solar cells and other fields.