A simulation and experiment were performed to demonstrate that a laser using volume Bragg grating as one of the cavity mirrors can achieve lasing even if the laser cavity length exceeds the traditional stable cavity condition. The laser transverse mode changes from a Gaussian beam into a ring-shaped mode as the laser cavity length increases from stable to unstable cavity conditions. At the same time, the effective modal reflectivity is reduced as the cavity length increases.