Machine-learning (ML) potentials are promising in atomic simulations due to their comparable accuracy to density functional theory but much lower computational cost. The descriptors to represent atomic environments are of high importance to the performance of ML potentials. Here, we implemented the descriptor in a differentiable way and found that ML potentials with optimized descriptors have some advantages compared with the ones without descriptor optimization, especially when the training dataset is small. Taking aluminum as an example, the trained potentials with proper descriptors can not only predict energies and forces with high accuracy of the first-principles calculations but also reproduce the statistical results of dynamical simulations. These predictions validate the efficiency of our method, which can be applied to improving the performance of machine learning interatomic potentials and will also strongly expand its applications.