Engineering strong single-photon optomechanical couplings is crucial for optomechanical systems. Here, we propose a hybrid quantum system consisting of a nanobeam (phonons) coupled to a spin ensemble and a cavity (photons) to overcome it. Utilizing the critical property of the lower-branch polariton (LBP) formed by the ensemble-phonon interaction, the LBP-cavity coupling can be greatly enhanced by three orders magnitude of the original one, while the upper-branch polariton (UBP)-cavity coupling is fully suppressed. Our proposal breaks through the condition of the coupling strength less than the critical value in previous schemes using two harmonic oscillators. Also, strong Kerr effect can be induced in our proposal. This shows our proposed approach can be used to study quantum nonlinear and nonclassical effects in weakly coupled optomechanical systems.