We propose and demonstrate an all-fiber passively Q-switched laser generating a third-order orbital angular momentum (OAM) pulse by introducing a few-mode long-period fiber grating (LPFG) into the laser cavity. The LPFG with asymmetric cross structure and strong refractive index modulation overcomes the coupling issue between the fundamental and the third-azimuthal-order (LP31 or OAM3) modes and realizes their direct conversion. A homemade graphene-based saturable absorber is used to realize Q-switched operation. The laser operates at a center wavelength of 1548.2nm, with a 3 dB spectral bandwidth of 0.4nm, and the OAM+3 and OAM-3 beams can achieve the purity of 90.0% and 90.2%, respectively. This all-fiber Q-switched laser has simple and compact structure and high purity of OAM±3 beams, which has potential applications in the fields of optical tweezers and material processing.