Suspensions of microswimmers in liquid crystals demonstrate remarkably complex dynamics and serve as a model system for studying active nematics. So far, experimental realization of microswimmers suspended in liquid crystalline media has relied on biological microorganisms that impose strict limitations on the compatible media and makes it difficult to regulate activity. Here, we demonstrate that acoustically powered bubble microswimmers can efficiently self-propel in a lyotropic liquid crystal. The velocity of the swimmers is controlled by the amplitude of the acoustic field. Histograms of swimming directions with respect to the local nematic field reveal a bimodal distribution: the swimmers tend to either fully align with or swim perpendicular to the director field of the liquid crystal, occasionally switching between these two states. The bubble-induced streaming from a swimmer locally melts the liquid crystal and produces topological defects at the tail of the swimmer. We show that the defect proliferation rate increases with the angle between the swimmer's velocity and the local orientation of the director field.