Optically levitated macroscopic objects are a powerful tool in the field of force sensing, owing to high sensitivity, absolute force calibration, environmental isolation, and the advanced degree of control over their dynamics that have been achieved. However, limitations arise from the spurious forces caused by electrical polarization effects that, even for nominally neutral objects, affect the force sensing because of the interaction of dipole moments with gradients of external electric fields. Here, we introduce a technique to measure, model, and eliminate dipole moment interactions, limiting the performance of sensors using levitated objects. This process leads to a noise-limited measurement with a sensitivity of 3.3 × 10-5 e. As a demonstration, this is applied to the search for unknown charges of a magnitude much below that of an electron or for exceedingly small unbalances between electron and proton charges.