Geometry optimization is a crucial step in computational chemistry, and the efficiency of optimization algorithms plays a pivotal role in reducing computational costs. In this study, we introduce a novel reinforcement-learning-based optimizer that surpasses traditional methods in terms of efficiency. What sets our model apart is its ability to incorporate chemical information into the optimization process. By exploring different state representations that integrate gradients, displacements, primitive type labels, and additional chemical information from the SchNet model, our reinforcement learning optimizer achieves exceptional results. It demonstrates an average reduction of about 50% or more in optimization steps compared to the conventional optimization algorithms that we examined when dealing with challenging initial geometries. Moreover, the reinforcement learning optimizer exhibits promising transferability across various levels of theory, emphasizing its versatility and potential for enhancing molecular geometry optimization. This research highlights the significance of leveraging reinforcement learning algorithms to harness chemical knowledge, paving the way for future advancements in computational chemistry.