Quantum fluctuations of a cavity field coupled into the motion of ultracold bosons can be strongly amplified by a mechanism analogous to the Petermann excess noise factor in lasers with unstable cavities. For a Bose-Einstein condensate in a stable optical resonator, the excess noise effect amounts to a significant depletion on long time scales.