Stimulated Brillouin scattering (SBS), a fundamental nonlinear interaction between light and acoustic waves occurring in any transparency material, has been broadly studied for several decades and gained rapid progress in integrated photonics recently. However, the SBS noise arising from the unwanted coupling between photons and spontaneous non-coherent phonons in media is inevitable. Here, we propose and experimentally demonstrate this obstacle can be overcome via a method called orbital angular momentum mode division filtering. Owing to the introduction of a new distinguishable degree-of-freedom, even extremely weak signals can be discriminated and separated from a strong noise produced in SBS processes. The mechanism demonstrated in this proof-of-principle work provides a practical way for quasi-noise-free photonic-phononic operation, which is still valid in waveguides supporting multi-orthogonal spatial modes, permits more flexibility and robustness for future SBS devices.