The up-conversion process is extensively studied because of its wide variety of applications such as bioimaging, energy harvesting, and optical sensors. However, the optical conversion efficiency is still relatively low and needs to be improved. Therefore, this paper introduces a detailed study of improving the up-conversion emission efficiency through adding plasmonic metallic nanostructures to the up-conversion optical centers. Our idea is to couple the optical plasmonic resonance with the visible emission of the optical centers under IR excitation. The optical centers are erbium ions hosted by fluoride low-phonon environment. Our calculations consider most possible transitions that can occur between the optical centers; tri-valent erbium ions, through Judd-Ofelt analysis. In addition, the effect of changing some parametric values is discussed, such as irradiance, and multi-phonon relaxations, to show their optimum values which correspond to best quantum yield efficiency. By increasing the diameter of added gold nanoparticles (Au NPs), the probability of occupation has been increased, and consequently, both the luminescence and up-conversion efficiency have been increased.