The impact of the environment on the properties of graphene such as strain, charge density, and dielectric environment can be evaluated by Raman spectroscopy. These environmental interactions are not trivial to determine since they affect the spectra in overlapping ways. Data pre-processing such as background subtraction and peak fitting is typically used. Moreover, collected spectroscopic data vary due to different experimental setups and environments. Such variations, artifacts, and environmental differences pose a challenge for accurate spectral analysis. In this work, we developed a deep learning model to overcome the effects of such variations and classify graphene Raman spectra according to different charge densities and dielectric environments. We consider two approaches: deep learning models and machine learning algorithms to classify spectra with slightly different charge densities or dielectric environments. These two approaches show similar success rates for high signal-to-noise data. However, deep learning models are less sensitive to noise. To improve the accuracy and generalization of all models, we use data augmentation through additive noise and peak shifting. We demonstrated the spectral classification with 99% accuracy using a convolutional neural net (CNN) model. The CNN model can classify Raman spectra of graphene with different charge doping levels and even subtle variations in the spectra of graphene on SiO2 and graphene on silanized SiO2. Our approach has the potential for fast and reliable estimation of graphene doping levels and dielectric environments. The proposed model paves the way for achieving efficient analytical tools to evaluate the properties of graphene.