We use magnetoconductance fluctuation measurements of phase-coherent semiconductor billiards to quantify the contributions to the nonlinear electric conductance that are asymmetric under reversal of magnetic field. We find that the average asymmetric contribution is linear in magnetic field (for magnetic flux much larger than 1 flux quantum) and that its magnitude depends on billiard geometry. In addition, we find an unexpected asymmetry in the power spectrum of the magnetoconductance with respect to reversal of magnetic field and bias voltage.