Interconversion between electron spin and other forms of angular momentum is useful for spin-based information processing. Well-studied examples of this are the conversion of photon angular momentum and rotation into ferromagnetic moment. Recently, several theoretical studies have suggested that the circular vibration of atoms work as phonon angular momentum; however, conversion between phonon angular momentum and spin-moment has yet to be demonstrated. Here, we demonstrate that the phonon angular momentum of surface acoustic wave can control the magnetization of a ferromagnetic Ni film by means of the phononic-to-electronic conversion of angular momentum in a Ni/LiNbO3 hybrid device. The result clearly shows that the phonon angular momentum is useful for increasing the functionality of spintronic devices.