As the field of optomechanics advances, quadratic dispersive coupling (QDC) represents an increasingly feasible path toward qualitatively new functionality. However, the leading QDC geometries generate linear dissipative coupling and an associated quantum radiation force noise that is detrimental to QDC applications. Here, we propose a simple geometry that dramatically reduces this noise without altering the QDC strength. We identify optimal regimes of operation, and discuss advantages within the examples of optical levitation and nondestructive phonon measurement.